DOCUMENT RESUME 



ED 460 971 



TM 033 647 



AUTHOR 

TITLE 



INSTITUTION 
SPONS AGENCY 
PUB DATE 
NOTE 

available from 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Bloom, Howard S.; Ham, Sandra; Melton, Laura; O'Brien, 
Julieanne 

Evaluating the Accelerated Schools Approach: A Look at Early 
Implementation and Impacts on Student Achievement in Eight 
Elementary Schools. 

Manpower Demonstration Research Corp., New York, NY. 

Ford Foundation, New York, NY. 

2001 - 11-00 

127p.; With Fred C. Doolittle and Susan Kagehiro. 

For full text: http://www.mdrc.org. 

Reports - Evaluative (142) 

MF01/PC06 Plus Postage. 

♦Academic Achievement; Educational Administration; 
♦Educational Change; Elementary Education; ♦Elementary 
Schools; Program Effectiveness; Program Evaluation; School 
Restructuring; Standardized Tests; Test Results 
♦Accelerated Schools; Reform Efforts 



ABSTRACT 



The Accelerated Schools Approach is a whole- school reform 
being implemented in more than 1,000 U.S. elementary and middle schools. The 
program is characterized by a school structure that sets high expectations 
for students and teachers, a governance structure with broad teacher 
participation, and a "powerful learning" approach to curriculum and 
instruction. This report looks at the implementation of the program in eight 
elementary schools and evaluates program effects at the third grade. An 
interrupted time series technique was used to study the effectiveness of the 
Accelerated Schools model . In the first 3 years of implementation, schools 
focused on reforming school governance and culture, turning to curriculum and 
instruction only in the third or fourth year. The findings indicate that the 
Accelerated Schools model improved standardized test scores in reading and 
mathematics once the schools in the study turned to changing curriculum and 
instruction. These findings should be interpreted with caution because of the 
small number of schools examined and the focus on a single grade. Impacts 
were not uniform across all students or all schools. The largest impacts were 
observed among students who would have scored in the middle of their school's 
test score distribution without the reform and among schools that had the 
lowest scores before the reform began. Two appendixes contain supplementary 
tables and the qualitative data protocols. (Contains 6 figures, 18 tables, 
and 42 references.) (SLD) 
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SUMMARY OF 

Evaluating the Accelerated Schools Approach 

A Look at Early Implementation and Impacts on Student Achievement in Eight 
Elementary Schools 

Howard S. Bloom, Sandra Ham, Laura Melton, and Julieanne O’Brien 

A key challenge faced by elementary schools is finding effective ways to help all students develop the skills 
needed to succeed in later education and the labor market. Remedial education programs have been a traditional 
response to academic problems, especially among poor and minority students, but past research has shown that 
this approach can keep students from joining the educational mainstream. The Accelerated Schools approach 
charts a different course, seeking to “accelerate” rather than slow down the learning of children at risk of school 
failure. 

Developed by Dr. Henry M. Levin and his colleagues at the Accelerated Schools Project (ASP), the Acceler- 
ated Schools model is now being used in more than 1,000 elementary and middle schools. Accelerated Schools 
seek to (1) create a new, supportive school culture that sets high expectations for teachers and students; (2) in- 
stitute a governance structure characterized by broad staff participation in decision making and by procedures 
for taking stock of the school’s strengths and problems and for generating solutions; and (3) introduce a “pow- 
erful learning” approach to curriculum and instruction that is more challenging, interactive, project-based, and 
relevant for students than traditional approaches. 

At ASP’s invitation and with funding from the Ford Foundation, MDRC conducted an independent evaluation 
of the Accelerated Schools reform in eight elementary schools around the country. The schools selected by the 
research team served a high proportion of at-risk students, had implemented the early version of the reform’s 
main components by the early 1990s, did not institute other major reforms during the study period, and were 
able to supply the requisite data. In the study’s “interrupted time series” design, the schools’ third-grade test 
scores in reading and math during the three years before the reform was launched were used to predict what the 
third-grade test scores would have been without the reform during each of the five following years. To estimate 
impacts on student achievement, the predicted test scores were then compared with the actual scores. The study 
focused on successive cohorts of third-graders because this grade marks a critical point in the development of 
basic reading and math skills. 

Key Findings 

■ During the first three years of implementation, the schools focused on reforming school governance and culture, 
turning to curriculum and instruction only in the third or fourth year. 

■ The reform’s impacts on third-grade test scores tracked schools’ implementation of it. There were no positive 
impacts in the first two years, a slight decline in the third year — as schools began to modify their curriculum and 
instruction — and a gradual increase in the fourth and fifth years. The average third-grade reading and math 
scores in the fifth year exceeded the predicted levels by a statistically significant amount. 

■ These impacts were not uniform across all students or all schools. The largest impacts were observed among stu- 
dents who would have scored in the middle of their school’s test score distribution without the reform and among 
the schools that had the lowest test scores before launching the reform. 

These results should be interpreted with caution for several reasons: They are based on a sample of only eight 
schools, the positive impacts took four to five years to emerge, and it is not known whether the impacts will 
persist in later grades. Nevertheless, these findings show that the Accelerated Schools approach improved aca- 
demic achievement in a group of mostly at-risk students. 

o . 4 
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Overview 



A key challenge faced by elementary schools is finding effective ways to help all students de- 
velop the skills needed to succeed in later education and the labor market. Remedial education programs 
have been a traditional response to academic problems, especially among poor and minority students, but 
past research has shown that this approach can keep students from joining the educational mainstream. 
The Accelerated Schools approach charts a different course, seeking to “accelerate” rather than slow 
down the learning of children at risk of school failure. 

Developed by Dr. Henry M. Levin and his colleagues at the Accelerated Schools Project (ASP), 
the Accelerated Schools model is now being used in more than 1,000 elementary and middle schools. Ac- 
celerated Schools seek to (1) create a new, supportive school culture that sets high expectations for teach- 
ers and students; (2) institute a governance structure characterized by broad staff participation in decision 
making and by procedures for taking stock of the school’s strengths and problems and for generating so- 
lutions; and (3) introduce a “powerful learning” approach to curriculum and instruction that is more chal- 
lenging, interactive, project-based, and relevant for students than traditional approaches. 

At ASP’s invitation and with funding from the Ford Foundation, MDRC conducted an independ- 
ent evaluation of the Accelerated Schools reform in eight elementary schools around the country. The 
schools selected by the research team served a high proportion of at-risk students, had implemented the 
early version of the reform’s main components by the early 1990s, did not institute other major reforms 
during the study period, and were able to supply the requisite data. In the study’s “interrupted time series” 
design, the schools’ third-grade test scores in reading and math during the three years before the reform 
was launched were used to predict what the third-grade test scores would have been without the reform 
during each of the five following years. To estimate impacts on student achievement, the predicted test 
scores were then compared with the actual scores. The study focused on successive cohorts of third- 
graders because this grade marks a critical point in the development of basic reading and math skills. 

Among the study’s key findings: 

• During the first three years of implementation, the schools focused on reforming 
school governance and culture, turning to curriculum and instruction only in 
the third or fourth year. 

• The reform’s impacts on third-grade test scores tracked schools’ implementa- 
tion of it. There were no positive impacts in the first two years, a slight decline in 
the third year — as schools began to modify their curriculum and instruction — 
and a gradual increase in the fourth and fifth years. The average third-grade 
reading and math scores in the fifth year exceeded the predicted levels hy a sta- 
tistically significant amount. 

• These impacts were not uniform across all students or all schools. The largest 
impacts were observed among students who would have scored in the middle of 
their school’s test score distribution without the reform and among the schools 
that had the lowest test scores before launching the reform. 

These results should be interpreted with caution for several reasons: They are based on a sample of only 
eight schools, the positive impacts took four to five years to emerge, and it is not known whether the im- 
pacts will persist in later grades. Nevertheless, these findings show that the Accelerated Schools approach 
improved academic achievement in a group of mostly at-risk students. 
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Preface 



This report brings new techniques and new findings to an education policy debate that is 
focused on two critical questions. First, can the standards of evidence for judging the 
effectiveness of education reforms be raised to help policymakers and program funders as they 
make decisions based on research findings? Second, can “whole-school” reforms — education 
interventions aimed at transforming schools on all fronts — improve student achievement? 

The Accelerated Schools approach is an influential whole-school reform targeted at 
students at risk of underachievement. This study owes much to the reform’s developer. Dr. 
Henry Levin, for his willingness to subject his work to an independent evaluation and to the Ford 
Foundation’s support of such essential research. To measure the reform’s effects on student 
achievement, the study used “interrupted time series,” a technique that — though rarely used in 
education research to date — shows potential for wide application in settings where randomized 
experiments are not feasible. It is hoped that broadening and strengthening the techniques for 
measuring program impacts, as this study does, will give rise to fruitful new lines of research on 
education reforms. 

The study also shows that implementing the Accelerated Schools approach improved 
standardized test scores — once schools turned to changing curriculum and instruction. The 
findings need to be interpreted cautiously because of the small number of schools examined and 
the study’s focus on just one pivotal stage of elementary schooling (the third grade). But they 
make a key contribution to the effort to learn whether whole-school reforms can raise student 
achievement and to discover how existing reforms might be improved to achieve this goal more 
effectively. 



Judith M. Gueron 
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Executive Summary 



Elementary schools are expected to teach children many skills that form the basis for suc- 
cess in later education and the labor market. Schools’ typical response to low academic achieve- 
ment, especially among poor and minority students, has been remedial education programs that 
slow down the pace of instruction or simplify the content of the curriculum. Past research sug- 
gests that this approach has serious limitations and can put students who are at risk of school 
failure at still greater disadvantage. 

The Accelerated Schools approach charts a different course, calling for a complex series 
of school changes designed to “accelerate” the learning of at-risk children. Developed over the 
last 15 years by Dr. Henry M. Levin and his colleagues imder the aegis of the Accelerated 
Schools Project (ASP), the approach is now in use in more than 1,000 elementary and middle 
schools. Accelerated Schools aim to (1) create a new, supportive school culture that helps all 
children learn and sets high expectations for teachers and students; (2) institute a new govern- 
ance structure consisting of a structured process for taking stock of the school’s problems and 
strengths, work groups (called cadres) devoted to school issues of concern, and school-wide 
meetings at which important decisions are reached by consensus; (3) use powerful learning, a 
teaching approach that is more challenging, interactive, project-based, and relevant for students 
than traditional instruction; and (4) receive technical assistance in implementing the reform from 
ASP’s National Center and 1 1 Satellite Centers across the coimtry. 

At ASP’s invitation, the Manpower Demonstration Research Corporation (MDRC) con- 
ducted an independent evaluation of Accelerated Schools at the elementary level. The objective 
of the study, which was funded by the Ford Foimdation, was to assess whether the Accelerated 
Schools approach improved student achievement in a small sample of schools that served at-risk 
students and had laimched the reform early in its development. This report, the culmination of the 
study, presents findings on the reform’s implementation and impacts. All eight schools in the 
study ^opted the reform in the early 1 990s, before powerful learning — a key component — 
was refined to make its implications for classroom practice clearer and before the technical assis- 
tance provided by ASP was extended and improved. The study examined the Accelerated 
Schools model’s effects on the reading and math achievement test scores of the schools’ third- 
grade cohorts during each of the five years after the reform began to be implemented. The third 
grade was chosen as the focus of the study because it marks a critical point in the development of 
basic reading and math skills and is late enough in elementary school for students to have been 
exposed to the reform for a considerable time (up to three years). 



I. 




Findings in Brief 

• Three years after laimching the Accelerated Schools model, all the study 
schools had adopted the practices aimed at improving school culture and gov- 
ernance. Changes in curriculum and instruction were implemented less sys- 
tematically and more slowly, with many schools starting to address these top- 
ics only in the third or fourth year of implementation. 

• At the end of the five-year follow-up period, the average third-grade math and 
reading test scores in the participating schools were higher than those during 
the baseline period by a statistically significant amoimt. The magnitude of 
these test score improvements is similar to that foimd in the well-known Ten- 
nessee class-size experiment. 
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• The findings suggest that students who would have been in the middle of their 
school’s test score distribution without the reform were the most likely to ex- 
perience an improvement in test scores. 

• The observed increases in average test scores were largest among the study 
schools that had the lowest average test scores before implementation of the 
Accelerated Schools model. 

• Improving test scores in schools that serve at-risk students has proven to be an 
extremely difficult challenge to meet, making the present findings of particular 
interest. Nevertheless, the effects found here should be interpreted with cau- 
tion because they are based on a sample of only eight Accelerated Schools that 
had reached an advanced stage of implementation by the early 1990s, did not 
emerge until four to five years after the reform was launched, and may not 
persist in later grades. 



II. Study Design 

To estimate the reform’s impacts on student achievement over a period of sufficient 
length, the schools in the study were required to have launched the Accelerated Schools reform at 
least five years before the start of the study period and to have five years of baseline test score 
data. ASP staff nominated 91 elementary schools that they believed had been implementing key 
elements of the reform for at least five years, and the ^ffiRC research team independently se- 
lected schools from this group. The eight urban and rural schools in the final sample were chosen 
for serving mostly at-risk students, having launched the reform and implemented its main com- 
ponents by the early 1990s, not having instituted other major reforms during the study period (to 
help rule out alternative explanations for any test score impacts observed), and being able to sup- 
ply the requisite data. It is important to bear in mind that the study schools, though not a hand- 
picked group selected on the basis of test score trends, are also not a representative sample of 
Accelerated Schools from the early 1990s. 

The study period encompassed a baseline (prelaunch) period of three years* and a follow- 
up (postlaunch) period of five years. The findings are based on an interrupted time-series design 
in which the test score level in each subject (reading and math) that would be expected in the ab- 
sence of the Accelerated Schools reform was estimated using test score data from the baseline 
period. Specifically, it was assumed that, if the reform had not been implemented, the test score 
level during the follow-up period would have been the same as the average level during the three 
baseline years. The difference between the expected level and the actual level in each follow-up 
year was taken as an estimate of the Accelerated Schools reform’s impact on test scores in that 
year.^ The impact analyses focus on the entire sample of schools rather than on individual 
schools. 



'impacts were also estimated using all five years of baseline data, but this procedure did not materially change 
the findings and was not used for reasons described in the following paper: Howard S. Bloom, “Measuring the Im- 
pacts of Whole-School Reforms: Methodological Lessons from an Evaluation of Accelerated Schools” (New York: 
MDRC, 2001). 

^he small changes in the composition of the student body that were observed, which might also have influenced 
test scores, were controlled for through statistical adjustments. 
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III. Implementation of the Accelerated Schools Approach 

• During their first three years of implementation, the study schools fo- 
cused on reforming school governance and culture; substantial changes in 
curriculum and instruction were typically not made until the third or 
fourth year. 

Most schools used the reform’s “inquiry” process to take stock of current school perform- 
ance and to identify key issues to address. They then used cadres to analyze courses of action and 
to develop recommendations, a steering committee to advise the cadres and coordinate the proc- 
ess, and school-wide meetings to decide important questions. The adoption of these processes 
was generally credited with creating an atmosphere of greater trust and support for staff, raising 
expectations of teachers and students, and increasing staff participation in decision-making. 

• Administrators and faculty working in these schools reported that when 
they flrst adopted the Accelerated Schools model, powerful learning was 
not precisely or concretely deflned. 

Many staff were confused about the instructional changes prescribed by the Accelerated 
Schools model. They also reported a lack of explicit guidance from ASP, especially about in- 
structional practices, after the start-up phase of the reform. During the first three years of the fol- 
low-up period, most of the schools used the governance procedures to reach their own decisions 
about instruction, with an emphasis on aligning the curriculum with new state or local standards 
rather than on developing new instructional techniques or new classroom practices. 



IV. Impacts on Student Achievement 

• Impacts tracked implementation. 

On average, there was no systematic change in test scores during the first two years of 
Accelerated Schools implementation, during which time the schools focused on establishing sup- 
portive cultures and decision-making processes. During the third year, test scores declined 
somewhat, as schools began to make instructional changes that may have temporarily disrupted 
instruction. During the last two years of the follow-up period, test scores gradually rose. 

• In the flfth follow-up year, the average test scores in the sample schools 
exceeded the baseline level in both reading and math by a statistically 
signiflcant amount. 

In the fifth year of the follow-up period, the average third-grade reading and math test 
scores in the study schools overall were 0.19 standard deviation and 0.24 standard deviation 
above their respective baseline averages. These differences, which are statistically significant, are 
small to modest by the conventional standards of evaluation research. However, the impacts 
found in the Tennessee class-size experiment — which are widely considered policy-relevant — 
are of similar magnitude. The positive impacts in the last follow-up year are especially notewor- 
thy given the low cost of this early version of the Accelerated Schools model relative to that of 
other school reform approaches. 

• The average reading and math scores in the study schools increased 7 to 8 
percentile points relative to the scores of students who took the same tests, 
leaving the study schools at or near the middle of their state or national 
distributions. 
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The positive impacts on test scores in the fifth follow-up year reflect an overall increase 
in the average reading score from the 37* percentile during the baseline period to the 44* percen- 
tile in the fifth follow-up year. The corresponding increase in the average math score was from 
the 45* percentile to the 53“** percentile. 

• The impacts were not uniform across students or schools. Students who 
would have been in the middle of their school’s test score distribution 
without the Accelerated Schools reform were the most likely to experience 
test score improvements. The schools that had the lowest test scores dur- 
ing the baseline period were the most likely to experience large impacts. 

The differential impacts on different groups of students may be attributable to the fact 
that the instructional changes made were not substantial enough to affect the lowest-achieving 
students. It is also possible that lower-performing students had higher mobility — that is, were 
more likely than higher-performing students not to have attended the Accelerated School during 
the entire follow-up period — and therefore received less exposure to the reform. Nevertheless, 
the Accelerated Schools initiative improved the performance of even lower-performing students, 
because (especially in the initially lowest-performing schools) even those students who scored in 
the middle of their school’s distribution were typically below average nationally or statewide. 
Furthermore, the schools that during the baseline period had the lowest test scores relative to 
their state or national norms experienced the largest relative gains. 



V. Implications for the Accelerated Schools Reform 

Together these findings demonstrate the potential of the Accelerated Schools approach, as 
it was implemented early in its development, to improve student achievement. Given the diffi- 
culty of raising test scores in schools like those in this study and the relatively low cost of Accel- 
erated Schools, these findings are noteworthy. 

The present study examined the effect of the Accelerated Schools model on the test 
scores of all third-grade students who attended the study schools during the follow-up period. 
Although the level of student mobility in these schools was relatively high, this focus was chosen 
because it addresses a policy-relevant question: What were the impacts of Accelerated Schools 
on third-grade students when the reform was implemented under real-world conditions — which 
include, among other things, high student mobility? Many other studies of education reforms, in 
contrast, have asked how the reform under study affects only those students who remain in the 
schools long enough to receive a full “dose” of the reform. Both questions are important, but the 
present findings may not be directly comparable to those from the latter type of research; specifi- 
cally, the impacts reported here are likely to be smaller than those found in other studies. 

These findings also suggest a need for some operational refinements of the Accelerated 
Schools approach, many of which have been put in place since the schools in this study began 
implementing the reform. The lack of emphasis on curriculum and instruction in the initial years 
of implementation, the negative impact on achievement in the third follow-up year, and the ap- 
pearance of positive impacts only at the end of the follow-up period all point to a need to focus 
on curriculum and instruction earlier in the reform process and to make powerful learning easier 
to implement. ASP now provides more technical assistance and encourages schools to change 
their curriculum and instruction earlier in the process. Further, ASP has worked extensively to 
make the concept of powerful learning more concrete — for instance, by providing staff with 
specific illustrations of ways in which they could modify their classroom practices. 
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Chapter 1 

Introduction 



Many poor and minority students leave public elementary schools without the academic 
skills needed to succeed in further education and to reach their full adult potential. Students who 
fall behind academically in elementary school have a difficult time catching up to the educational 
mainstream; for example, academic achievement in the third grade has been found to be corre- 
lated with success in high school and postsecondary schooling.' Traditional public schools in 
poor and minority communities often address the problem of underachievement through remedial 
programs that slow down the pace of learning or simplify the content of the curriculum. In some 
cases, this is in response to individual students who are found to have only rudimentary academic 
skills. In other cases, it arises from schools’ perception that poor and minority students as a group 
are slow achievers. 

Past research suggests that school strategies that rely on remediation and ability grouping 
may contribute to low academic growth and achievement in the long term^ and to differences in 
academic achievement along racial and socioeconomic lines. ^ The main disadvantage of this ap- 
proach is that children in low-achievement classes or schools are typically offered lower-quality 
instruction and curricula than other children. The resulting differences in learning opportunities 
may play an important causal role in the performance gap between low- and high-achieving stu- 
dents.^ There is strong evidence in the educational literature that participation in a demanding 
academic curriculum promotes academic success.^ In addition, the literature suggests that the de- 
gree of opportunity to learn rigorous academic subjects — for instance, in advanced placement 
courses — varies across socioeconomic and racial groups. Poor and minority students’ lesser ac- 
cess to beneficial learning opportunities begins early in their education, reducing the likelihood 
of their being exposed to an academic curriculum in high school that meets the entrance require- 
ments of most four-year colleges.® 

I. The Accelerated Schools Proiect (ASP) 

The Accelerated Schools approach is one of the nation’s best-known whole-school re- 
forms designed to improve the school performance of students at risk of underachievement. 
Henry Levin of Stanford University established the Accelerated Schools Project (ASP) to address 
two issues; traditional schools’ lack of success in educating at-risk students and the growing pro- 
portion of such students entering public schools. According to ASP, more than 1,000 elementary 
schools and middle schools in diverse communities in this country and abroad have adopted the 
Accelerated Schools approach, and ASP has 11 Satellite Centers that provide support to schools 
in this network. 



'Copple, Deich, Brush, and Hofferth, 1993; Carnegie Corporation, 1996. 

^Dougherty and Barth, 1997; Education Trust, 1996. 

^Oakes and Lipton, 1990. 

^Nettles, 1997; Glover and Marshal, 1992. 

^Annie E. Casey, 1997; Solorzano and Solorzano, 1995; Natriello, McDill, and Pallas, 1990. 
*Gamaron, 1987. 
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The Accelerated Schools approach calls for a complex series of changes throughout a 
school that are intended to accelerate the learning of at-risk children. These changes, described 
more fully in Chapter 2, can be grouped into four areas. First, Accelerated Schools seek to create 
a new organizational culture committed to helping all students learn and succeed in school, set- 
ting high expectations for teachers and students, and creating a supportive and trusting environ- 
ment in the school. Second, Accelerated Schools institute a new governance structure designed to 
support a far-reaching assessment of the school’s current performance and areas in need of im- 
provement as well as the development of potential solutions. As part of this process. Accelerated 
Schools open the door to broader and more egalitarian staff participation in decision-making 
through new bodies — such as “cadres” that address specific issues, a “steering committee,” and 
meetings that include the entire staff and at which all important decisions are made. Third, Ac- 
celerated Schools seek to introduce a teaching approach that is intended to be more challenging, 
interactive, project-based, and relevant for students than traditional instruction. Finally, ASP’s 
National Center and Satellite Centers provide technical assistance to schools, helping them use 
the reform’s approach to improve their schools. The Satellite Centers train school staff and 
committees at the launch of the reform and provide schools with ongoing support and networking 
opportunities. ASP has continued to refine its definition of and support for the reform, most im- 
portantly by seeking to strengthen aspects of instruction and curriculum and the technical assis- 
tance provided to schools. 

The national prominence of Accelerated Schools as a reform is partly attributable to its 
low out-of-pocket cost relative to that of other whole-school reforms.’ Its primary costs are that 
of hiring a part-time staff person from the school or district to serve as a “coach” during the im- 
plementation process and that of training local teachers and school staff. The rest of the labor and 
materials required by the reform are expected to be made available through reallocation of exist- 
ing resources. Although little detailed information about the costs of implementing Accelerated 
Schools is available from independent sources, one study estimated that the first-year cost of im- 
plementing the approach ranges from a low of $34 per student for the “basic model” to a high of 
$160 per student.^ To put these costs in perspective, the author also provided estimates of the 
costs of implementing two other major comprehensive school reform models — Robert Slavin’ s 
Success for All, a reading program for elementary school students; and James Comer’s School 
Development Program, a whole-school reform. The cost estimates for Success for All range from 
$321 to $681 per student,’ , and those for the School Development Program range from $115 to 
$438 per student. 

II. An Evaluation of Accelerated Schools 

The Manpower Demonstration Research Corporation (MDRC) was invited by ASP to 
conduct an independent evaluation of a sample of elementary schools implementing the Acceler- 



’To get a complete picture of a reform’s cost, one would compare the marginal costs of implementing it (the value 
of the net additional resources that it requires) with the marginal benefits that it brings (its net impacts) on a common 
accounting basis (for example, per student). In this case, such a thorough analysis is not possible for lack of data. 

*Bamett, 1996. 

’These findings were obtained by dividing the total costs of Success for All listed in Table 9-2 of Barnett (1996) 
by the number of students per school (500) used by the author to estimate the total costs presented in the table. 
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ated Schools reform. To ensure the independence of the evaluation, ASP requested that MDRC 
raise the funds needed to conduct it. The study, which was supported by the Ford Foundation, 
had two goals: (1) to develop better methods for assessing the effects of whole-school reforms 
such as Accelerated Schools and (2) to use these new methods to estimate the effects of the Ac- 
celerated Schools reform on student achievement in a sample of “mature” Accelerated Schools at 
the elementary level. Here maturity is meant to indicate that the schools were experienced in im- 
plementing the reform, not that they implemented the reform optimally or in a way that was rep- 
resentative of Accelerated Schools at the time. The present work was intended to be an explora- 
tory first stage in a possible larger project that would apply the research methodology introduced 
here to a sample of schools that is both larger and more representative of schools implementing 
the current version of the model. 

The goal of research on the effects of whole-school reforms such as Accelerated Schools on 
student achievement is to estimate the difference that the reform makes by comparing the outcomes 
for students attending schools that have implemented the reform with the outcomes that students 
would have experienced in the absence of the reform. It is often straightforward to document out- 
comes such as achievement and attendance for students attending schools that have implemented a 
reform. But to understand the difference a reform makes, it is necessary to compare these outcomes 
with those that would have occurred under the previous or usual education program; the differences 
between the two serve as estimates of the “impacts,” or effects, of the program. 

The several studies of Accelerated Schools conducted to date employed a variety of 
methods for estimating impacts on student achievement. Most of them examined a single 
school and used the outcomes for students in the same school in the year prior to implementation 
of the reform as the benchmark — referred to as the “counterfactual” in the language of evalua- 
tions — against which to compare students’ achievement after the reform was launched. Other 
studies used as the counterfactual the outcomes for students in demographically similar schools 
that did not implement the reform.'' 

A. Goals of the Study 

The study design was developed with three considerations in mind. First, it was important 
to assess the impacts of the Accelerated Schools reform only in schools that had implemented the 
key elements of the model — that is, in mature Accelerated Schools rather than in schools that 
were in the midst of launching the reform or were known to have encountered serious problems 
in sustaining implementation of it; at the same time, the schools included in the study were not 
hand-picked for being particularly successful in implementing the approach. Second, it was im- 
portant to attempt to rule out alternative explanations for the impacts observed. For example, the 
study would have been compromised if other reforms or important school changes, such as new 
achievement standards or an entirely new curriculum, occurred simultaneously with implementa- 
tion of the Accelerated Schools reform. These first two points were addressed through the proce- 
dures used to select the study schools and to conduct field research on the extent of implementa- 

'°For a review of this literature, see “Research Background on Accelerated Schools” and “Accomplishments of 
Accelerated Schools” at the Accelerated Schools Project Web site: http://www.acceleratedschools.net. 

"in both cases, statistical techniques can be used to adjust for observed differences in characteristics between 
comparison students and students in the reformed schools. 
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tion in those schools. Third, it was necessary to identify a group of students who did not receive 
an Accelerated Schools education who could be compared with students in the Accelerated 
Schools in the study. This last goal was realized using a “backward-looking interrupted-time- 
series” methodology in which the counterfactual is the outcomes for students who attended the 
study schools before the reform was implemented. 

It should be emphasized that the study was designed to estimate the impacts of the Accel- 
erated Schools reform on student achievement — as reflected in scores on standardized tests of 
reading and math — across all eight schools selected for inclusion in the study sample rather than 
at the level of individual schools. The methodological advantages of this approach are discussed 
in Chapter 5. 

B. Focus on Student Achievement 

The present analysis focuses on reading and math achievement in the third grade because, 
as already mentioned, achievement in the early school years is a key predictor of subsequent edu- 
cational success.*^ By necessity, this analysis is based on students’ scores on the standardized 
tests administered in the study’s Accelerated Schools. It is important to acknowledge, however, 
that the Accelerated Schools reform is not targeted at test scores per se but rather at a broad range 
of student outcomes. 

As discussed in more detail later in this report, the analysis depends on the following five 
steps or conditions: (1) careful selection of a sample of schools that met the criteria outlined 
above, (2) availability of student achievement data for multiple years prior to implementation of 
the Accelerated Schools approach to serve as the counterfactual, (3) availability of data on stu- 
dents’ demographic characteristics to allow changes in the composition of the student body over 
time to be controlled for, (4) field research to assess the extent of implementation of the Acceler- 
ated Schools reform and other education reforms, and (5) examination of whether student 
achievement levels broke with previous trends after the reform was launched. 

C. Study Timeline 

Because this report often refers to events by the years in which they occurred, it is impor- 
tant to establish a clear chronology. The eight years encompassed by the analysis are shown in 
Figure 1.1. The three years prior to the beginning of Accelerated Schools implementation are the 
baseline years, starting with baseline year 1. The year in which a school voted to adopt the Ac- 
celerated Schools reform is follow-up year 1, after which come follow-up years 2 through 5. 

The question that the study addresses is the following: Following implementation of the 
Accelerated Schools approach (as the reform was implemented in the mid 1990s), did the reading 
and math test scores of third-graders in the study schools increase relative to the levels that 
would be expected given the past scores of third-graders in the schools? 

In addressing this question, the present analysis focuses on the last two years of the 
study’s five-year follow-up period because of the process-oriented nature of the Accelerated 



'^Given the limited resources available for this study and the difficulty of obtaining historical data in some 
schools, it was not possible to obtain test scores for more than one grade. 
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Accelerated Schools Evaluation 
Figure 1.1 
Timeline 



Baseline Period Follow-Up Period 




Adoption of Reform 



Schools reform. Another reason for focusing on these years is that, with each successive year of 
implementation, students who have continuously attended the study schools will have received 
more years of Accelerated Schools education prior to reaching the third grade. For example, 
whereas in the first year of implementation third-graders could not have received more than one 
year of an Accelerated Schools education, by the third year of implementation they could have 
received up to three years. Because the probability of observing impacts presumably increases 
with students’ degree of exposure to the reform over time, the later years of implementation 
should be more revealing than the earlier years. 

D. Student Mobility 

The analysis is based on the test scores for students who took the third-grade tests in each 
year, whether or not these students had continuously attended the school in earlier grades or even 
for their entire third-grade year. Student mobility in the type of schools likely to implement the 
Accelerated Schools reform tends to be high, so the cohorts of third-grade test-takers in the pre- 
sent sample includes students who did not get a full “dose” of Accelerated Schools over their 
years of schooling. Chapter 5 examines the extent of student mobility and how this should affect 
the interpretation of impacts. 

An alternative approach would have been to limit the analysis to students who stayed in 
the study schools from the first grade through the third grade. There are two problems with this 
approach. First, it is difficult to identify the appropriate comparison group for this group of stu- 
dents, especially if the reform makes the schools more attractive and thereby decreases student 
mobility. Past research suggests that students who change schools tend to have lower test scores, 
so it would not be appropriate to compare them with all students attending the school in prior 
years.'^ Second, the fact that student mobility is often high in schools that have implemented the 



'^he same problem would arise if the design used comparison schools. Some analysts have compared the ex- 
periences of students who start a particular grade in an Accelerated School and continue to attend that school for 

(continued) 
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Accelerated Schools approach makes a focus on reform impacts for those students who get a full 
dose less policy-relevant. The real question facing schools is whether a reform can help the stu- 
dents they serve (for whatever period of time) learn more and be more successful in school and 
later life. It is important to note that the key research question in this context differs from that in 
some other evaluations of school reforms, a difference that should be borne in mind when com- 
paring results from different evaluations. 

E. Scope of the Study 

This evaluation takes a first step toward understanding the effects of the Accelerated 
School approach on student achievement in a sample of schools that were experienced in using 
the model early in its development. To permit generalization of the present findings to the Accel- 
erated Schools approach as a whole, it will be important to look at a larger and more diverse 
sample of schools; to include schools that began to implement the reform more recently, after 
aspects of the model and its implementation changed; and to refine the basic impact methodology 
as suggested by the present work. 

F. Overview of This Report 

Subsequent chapters of this report present the Accelerated Schools approach in more 
detail (Chapter 2), describe how schools were selected for inclusion in the study and what the 
student body characteristics were in the selected schools (Chapter 3), assess the educational 
changes produced by the adoption and implementation of the Accelerated Schools approach in 
the study schools (Chapter 4), and present estimates of the impacts of this reform on student 
achievement (Chapter 5). The final chapter (Chapter 6) summarizes the conclusions and im- 
portant constraints on their interpretation. 



several years with those of students who start the same grade in a comparison school and continue in that school for 
several years. It is likely that the two groups will become less comparable over time, especially if student mobility is 
affected by the reform. 
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Chapter 2 

The Accelerated Schools Approach to School Reform 



The Accelerated Schools approach aims to transform traditional schools, where at-risk 
students typically receive remedial education, into schools that provide all students with enriched 
academic experiences — with the ultimate goal of improving the achievement of all students.* 
The Accelerated Schools approach rests on the belief that at-risk children are most likely to 
“catch up” to the educational mainstream when they are placed in learning environments where 
expectations are high and challenging materials are introduced at a fast pace. 

Unlike many school reforms, the Accelerated Schools approach does not demand that 
schools adopt prescribed curricula or pedagogical techniques in order to provide students with 
these enriched academic experiences. Instead, each Accelerated School involves the whole 
school community in an effort to develop a plan for reform that builds on the community’s 
unique qualities, issues, and strengths. In addition, a new governance structure is established that 
empowers teachers and students to participate fully in the school decision-making process and to 
share responsibility and accountability for the outcomes of these decisions. 

The purpose of this chapter is to describe the aspects of the Accelerated Schools approach 
that are related to student achievement and that can be systematically documented as evidence of 
the reform’s implementation. The approach encompasses four main areas: governance and man- 
agement tools, school culture, curriculum and instruction, and technical support. This chapter 
will describe each area, beginning with a brief history of ASP. 



I. History of ASP 

Henry M. Levin, currently the William Heard Kilpatrick Professor of Economics and 
Education at Teachers College, Columbia University, founded the Accelerated Schools approach 
in 1986. In that year, two pilot Accelerated Schools (both at the elementary level) were launched 
in California.^ The Accelerated Schools approach has since been adopted by more than 1,000 
elementary and middle schools nationwide. The model has not only grown more widespread 
over the years, but its components have evolved as well. Several examples are provided below. 

• Originally the training provided by ASP placed heavy emphasis on using di- 
rect instruction methods to train teams of administrators, teachers, staff, and 
parents to implement the model at their schools. Little follow-up training or 
support was provided. In 1992, ASP began to use more interactive methods of 
training to train “coaches” instead of school staff. 

• The concept of powerful learning is at the center of the Accelerated Schools 
approach, as reflected by the fact that the 1993 edition of the Accelerated 



'Chasin and Levin, 1995; Levin, 1987a; Levin, 1987b. 

^Accelerated Schools Newsletter, Fall and Winter 1996-1997, Volume 6, Number 1. 
^See http://www.stanford.edu/group/asp. 
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Schools Resource Guide^ devoted four chapters to the subject. Nevertheless, 
many Accelerated Schools had difficulty employing powerful learning consis- 
tently. From 1994 to 1996, the National Center worked to flesh out the defini- 
tion of powerful learning and the activities involved in putting it in place (de- 
tails are provided later in this chapter). 

• In the beginning, ASP’s primary focus was on specifying the governance 
structures necessary to effect whole-school change. Powerful learning was 
therefore not emphasized until the intended changes in the school’s govern- 
ance structure had occurred. As ASP examined ways to improve the definition 
of powerful learning, it also strove to discover better ways to implement it. In 
one key change in strategy, schools were encouraged to implement powerful 
learning alongside the changes in governance. 

This chapter will describe the current version of the Accelerated School model, although 
the schools included in the present evaluation were among the first to adopt the model and there- 
fore employed earlier versions of it. 



II. Governance Structure and Management Tools 

Central to the Accelerated Schools approach is a shared governance structure and a set of 
management tools that involve administrators, teachers, students, parents, and community mem- 
bers taking stock of their current status and setting priorities for action. The changes in govern- 
ance and management described in this section are designed to lead to and reinforce the changes 
in school culture discussed in the next section. 

A. Governance Structure 

The governance structure is the most prescribed aspect of the Accelerated Schools ap- 
proach. For this approach to governance to work, a school must agree on a set of goals for the 
whole school community. All reform efforts should be aimed at the realization of these goals, 
and the school community should be actively involved in guiding the school toward this realiza- 
tion. Furthermore, the Accelerated Schools approach holds that reform efforts should build on 
the existing resources and expertise of the school community. The governance structure encom- 
passes three bodies: 

• Cadres are task forces or committees that focus on specific issues or areas of 
priority for the school, such as improvements in school facilities or instruc- 
tion. These committees are responsible for assessing their priority area and 
developing an action plan that addresses the issues. 

• The steering committee is a leadership team that advises the cadres and is re- 
sponsible for coordinating the functions of the cadres and the meetings of the 
entire school. 



^Hopfenberg et al., 1993. Unless otherwise noted, the information about the Accelerated Schools approach pre- 
sented in this chapter is drawn from this source. 
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• The school-as-a-whole (SAW) is a group that consists of the entire school 
body: students, faculty, staff, parents, and community representatives. 

Ideas about how to change the school flow from the cadres through the steering commit- 
tee to the SAW. Specifically, the cadres develop action plans that are reviewed by the steering 
committee. If an action plan passes this review, cadres present their plans to the entire school 
body at an SAW meeting. Final approval of the action plan is reached through consensus at the 
SAW meeting. 

B. Management Tools 

The Accelerated Schools approach provides schools with tools for implementing and 
maintaining the model. The management tools are introduced during the school’s first year of 
implementation in the following five stages:^ 

1. Taking stock. Taking stock is used to build the school’s reform agenda. In this 
stage, the school community comes together to assess its needs and to identify 
its resources. Results of the taking-stock process are compiled in a report. 

2. Development of a collective school vision. In this stage, the school community 
collectively decides where it wants the school to go and defines a vision for 
the school. Involving all members of the school community fosters a feeling of 
ownership that in turn is more likely to result in action. 

3. Identifying priority or challenge areas. Given the school’s current status and 
the school’s vision of where it wants to be, the community then identifies its 
three to five greatest challenges. Prioritization is essential but also potentially 
difficult, because schools may face numerous challenges. 

4. Implementing the Accelerated Schools governance structure. The next step is 
to implement the governance structure. Here the goal is to set up the cadres to 
focus on the priority areas identified in the preceding stage and to establish a 
steering committee to oversee and coordinate the cadres’ efforts. 

5. Inquiry process. In the last stage, the school community begins to tackle the 
challenges using a highly prescribed problem-solving process. Although all gov- 
ernance groups, grade levels, departments, and individuals in an Accelerated 
School use the inquiry process to guide all decision-making and communication 
in the school, this section will illustrate the inquiry process using a single case 
study from the Accelerated Schools Resource Guide, a manual that expounds the 
Accelerated Schools philosophy. The inquiry process consists of five steps: 

• Assess the problem. Each cadre focuses on a challenge area identified 
in the taking-stock phase. In one school in the present study, a cadre 
called the Family Involvement Cadre was established to concentrate on 
boosting family involvement in the school. Cadre members are en- 
couraged to state informal h)^otheses or assumptions concerning why 



^Hopfenberg et al., 1993. 
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a particular problem exists. The Family Involvement Cadre, for exam- 
ple, hypothesized that language barriers and lack of trust prevented 
families from getting involved. Cadres then test these assumptions to 
see if they are accurate explanations. To find out if issues of language 
and trust were responsible for low family involvement, the Family In- 
volvement Cadre hosted focus groups in which a diverse set of fami- 
lies were asked to express their views and feelings about the school. 
By assessing the problem, cadres try to develop a clear understanding 
of the challenge areas. 

• Brainstorm solutions. Cadres generate possible solutions to problems 
by brainstorming, looking both inside and outside the school for ideas. 
The Family Involvement Cadre talked to other schools, conducted lit- 
erature searches, and considered ideas generated in the focus groups. 

• Synthesize solutions and develop an action plan. The cadres then 
check to make sure that the solutions fit the needs of their school and 
are feasible given the school’s strengths and resources. Each cadre de- 
cides who will do what tasks by what dates and how the tasks will be 
performed. The Family Involvement Cadre decided to create a family 
room and a family newsletter and made plans for obtaining resources 
and staff for these initiatives. 

• Pilot-test or implement the action plan. After securing approval for the 
action plan from the steering committee and the SAW, cadres work 
with the steering committee to set a timeline and allocate responsibility 
for implementation, documentation, and assessment of the action plan. 
An open house held by the Family Involvement Cadre was poorly at- 
tended, prompting the cadre to consider how to inform more parents 
about the room; the newsletter, in contrast, was an instant success and 
served to get more parents involved in the school. 

• Evaluate and reassess. In this last phase, cadres use data gathered dur- 
ing the pilot test or early phase of implementation to judge the success 
of the action plan in addressing the initial problem. This step could 
lead to full-scale implementation, revision, or abandonment of the ac- 
tion plan. The Family Involvement Cadre revised its strategies for get- 
ting parents to use the family room and decided to implement the 
newsletter with regular features and a regular distribution schedule. 



III. School Culture 

School culture is defined by, among other things, how comfortable students and parents 
feel at school, how students behave, whether teachers feel free to try ideas, and whether all cul- 
tures are validated. The Accelerated Schools approach is relatively prescriptive with respect to 
what a school culture should include. The model outlines three central principles and nine values 
designed to drive cultural change in Accelerated Schools. Acting on these principles via the gov- 
ernance structure described in the previous section is intended to enable a school to respond to its 
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community’s specific issues and needs, make the school community’s efforts toward change 
more cohesive, and ensure that the change is sustainable. The three principles are: 

1. Unity of purpose: the striving of the entire school community toward a com- 
mon set of goals and expectations for the school 

2. Empowerment coupled with responsibility: the ability of the school commu- 
nity to share responsibility for decision-making, implementation of change, 
and outcomes 

3. Building on strengths: using all available resources, including students, par- 
ents, district and school staff, and community members 

The following nine interrelated and interdependent values guide actions and interactions 
in the school community and are intended to ensure that the school culture fosters growth, crea- 
tivity, and enrichment. The nine values are: 

1. Equity. All students in the school have an equal right to an enriched education. 

2. Participation. All members of the school community — including students, 
teachers, parents, administrators, and support staff — participate in the school. 

3. Communication and collaboration. The school community works toward 
common goals through active communication. 

4. Community spirit. Members of the school community are connected to one 
another, and achievements are celebrated as community accomplishments. 

5. Reflection. Time must be set aside for reflection, research, and sharing ideas. 

6. Experimentation. The school community fosters a spirit of experimentation 
and discovery among its members. 

7. Trust. All members of the school community must believe in and support each 
other. 

8. Risk-taking. The school community encourages its members to be entrepre- 
neurial and expects that some programs will succeed and others will fail. 

9. School as center of expertise. The school is the center of the community, and 
its members are the experts who know how to respond to challenges and cre- 
ate the best learning environment for students. 



IV. Curriculum and Instruction 

The Accelerated Schools approach provides a framework for instruction known as powerful 
learning. Powerful learning is based on a constructivist approach in which students acquire or con- 
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struct new understanding through concrete experience, collaborative discourse, and reflection.* *^ In 
constructivist learning, students develop the skills to organize, synthesize, interpret, explain, and 
evaluate information as a means of buildng new knowledge based on their prior experiences. 

The Accelerated Schools approach defines powerful learning experiences as those in 
which students perceive meaning and relevance in their work and are motivated to learn. Tradi- 
tionally, this type of curriculum and instruction has been offered only to children thought to be 
gifted or talented. While powerful learning can take place in a variety of settings — such as in 
the classroom, at home, or on the playground — the Accelerated Schools model requires that it 
be part of every classroom experience rather than a departure from the regular curriculum. The 
components of powerful learning experiences are as follows: 

• Authentic: relevant to students’ lives 

• Interactive: based on the use of exploratory techniques 

• Learner-centered: focused on students’ needs 

• Inclusive: sensitive to different learning styles and cultural backgrounds 

• Continuous: built on prior knowledge 

The following is an example of powerful learning. One Accelerated Schools classroom 
had a diabetic student. Students in the class were curious about diabetes, and the teacher re- 
sponded to their curiosity by asking them what they wanted to leam and calling on specialists 
from the school and the community to discuss their questions. Prompted by additional questions, 
the students also studied a skeleton, visited a blood laboratory at a local hospital, studied the 
Latin roots of medical terms, and learned how to read blood test results. This learning experience 
was authentic because students generated the questions. It employed interactive techniques dur- 
ing the visit to the hospital. The lesson was learner-centered in that it was guided by student 
questions and inclusive in that it involved all students and used a variety of teaching methods. 
Finally, the lesson was continuous because students continued to leam as new questions arose. 



V. Technical Support 

Becoming an Accelerated School is a challenging and time-consuming endeavor, but 
schools are supported in their efforts to launch and sustain the Accelerated Schools model. ^ A 
“coach” guides the school community through the initial phase of the reform, remaining with the 
school to provide ongoing support. The coach, in turn, is trained and supported by a regional 
ASP Satellite Center and the ASP National Center.^ The functions of these three sources of sup- 
port are outlined in detail below: 



*This definition and the following components and example of powerful learning are based on the Accelerated 
Schools Newsletter, Fall and Winter 1996-1997, Volume 6, Number 1. 

’This and the following paragraph about the coaching component are based on Finnan et al. (1995). 

*Since the reform was launched in the eight schools in the present study, the amount of training and technical as- 
sistance provided to Accelerated Schools has increased. 
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Accelerated Schools coach. Coaches work with schools to guide them through the proc- 
ess and to prepare them to sustain the momentum of transformation. Coaches come from a vari- 
ety of settings, including school districts, universities, and state agencies.^ Shortly after the 
school launches the reform, coaches work with the school approximately one day per week, but 
these visits typically become less regular as the school becomes accustomed to implementing 
Accelerated Schools processes and values. Coaches also participate in retreats and networking 
events where they can share their knowledge and help schools address new challenges. 

National Center for the Accelerated Schools approach. The National Center, located at 
the Neag School of Education at the University of Connecticut, directs all nation-wide Acceler- 
ated Schools activity — training coaches, supporting Satellite Centers, performing research, and 
disseminating information about the approach. 

Satellite Centers. Satellite Centers are regional offices that organize local networks of 
Accelerated Schools and are increasingly responsible for providing support and training services 
to build local capacity for implementing the reform. Owing to the rapid spread of the approach 
and the National Center’s belief that a regional office is better suited to maintaining contact with 
local coaches and schools, in recent years the National Center has relied increasingly on the Sat- 
ellite Centers." 



VI. Implications for This Evaluation 

The purpose of this evaluation was to estimate the impact of the Accelerated Schools ap- 
proach on student achievement. The model described in this chapter might affect student 
achievement in any of several ways — indirectly by transforming the school community into a 
dynamic learning environment or more directly by boosting test scores through implementation 
of powerful learning practices. Through high expectations and the rapid introduction of challeng- 
ing material, powerful learning is intended to speed up the pace of learning for all children. The 
other components of the Accelerated Schools model described in this chapter are designed to 
support and facilitate the powerful learning environment by strengthening the focus on student 
learning and raising expectations about what students can accomplish. 

The governance structure and management tools are designed to ensure that the changes 
in school culture and instruction will be long-lasting. The governance structure allows the school 
community to “own” the reform and thereby to take responsibility for its decisions. The man- 
agement tools and decision processes promote collaboration within the school community, allow 
the school to search widely for scarce resources, and promote flexibility in tackling the unique 
challenges faced by the school. The changes in school culture are intended to create an environ- 
ment where discipline problems decrease and attention can therefore be focused on learning. Fi- 
nally, the technical support is aimed at facilitating changes in long-held beliefs about education, 
modifying long-standing school practices, and ensuring that these modifications take hold. 



’Coaches do not come from the school itself. 

'°The National Center was located at Stanford University until early 2000. 
^^Accelerated Schools Newsletter, Fall and Winter 1996-1997, Volume 6, Number 1. 
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Chapter 3 

Site Selection 



The goal of site selection in this evaluation of Accelerated Schools was to find schools that 
had satisfactorily implemented the Accelerated Schools approach at the elementary level, met the 
requirements of the research design, and served students in the population targeted by the model. Each 
of these requirements is briefly summarized in this introductory section. 

The schools in the sample were not selected for being ideal or representative examples of the 
full set of schools that have adopted the model since its inception in 1986. Instead, as explained in 
Chapter 1, they were selected for having been mature Accelerated Schools in the early 1990s — that 
is, for having been experienced at implementing those of the model’s components that were 
emphasized early in the reform’s evolution, including those affecting governance, decision-making, 
and school culture.' 

In addition to having implemented the model, the candidate schools had to satisfy the 
requirements of the research design. These requirements included serving the appropriate grades, 
having a sufficiently large number of third-grade students, and having data available on students who 
attended the third grade in previous years. Moreover, for changes in test scores to be directly 
attributable to the Accelerated Schools approach, the schools had to have been fairly stable over the 
study period. That is, the demogr^hic composition of the student body had to have been fairly 
constant, and the school and district had to be fi’ee of major dismptions (such as a change in 
superintendent), both factors that could have affected student achievement and thus obscured the 
model’s effects. 

The retrospective nature of the research design allowed for the selection of schools for 
inclusion in the study after the reform had been in place for four to five years. This procedure reduced 
the evaluation’s vulnerability to imexpected changes within the school or district that might obscure 
the reform’s effects, because schools in which such changes had occurred could be excluded fixjm the 
sample. Nevertheless, schools are dynamic, and some degree of change fi’om year to year is natural 
and expected. The instability due to year-to-year variability in student characteristics such as ethnic 
composition was accommodated in the estimation of the model’s impacts on achievement. 

It is believed, especially with regard to schools that adopted the reform early in its 
development, that the effects of the Accelerated Schools reform on achievement might not be 
observed imtil three to five years after its laimch. After three to five years of implementation — 
having already modified school culture, governance, and decision-making as prescribed by the 
model — schools are expected to have acclimated to the reform, thus enabling them to turn their 
attention to improving curriculum and instruction. Therefore, at the beginning of the study the 
sample schools had to be able to supply the researchers with historical standardized test data for 
10 school years: the five years before laimch and the five years after launch. After site selection 
took place, it was decided that only three years of historical test data prior to the launch of the 



'The model was first pilot-tested in two schools in the 1986-1987 school year. By the fall of 1990, there were 
about 54 Accelerated Schools nationwide (Hopfenberg et al., 1993). 
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reform would be used (hence the three-year baseline period; for a thorough discussion of the 
research design and analysis, see Chapter 5). 

Finally, the schools in the study had to serve students in the population targeted by the 
Accelerated Schools model, that is, students at risk of educational failure. In its Resource Guided ASP 
defines at-risk students as “those who are unlikely to succeed in schools as schools are currently 
constituted because they bring a different set of skills, resources, and experiences than those on which 
school success is traditionally based” (p. 9). The guide goes on to note that “children in at-risk 
situations are disproportionately concentrated among families that are in poverty, families headed by 
single parents, non-English speaking families, minority families, and migrant families” (p. 10). 

The remainder of the ch^ter describes first the strategy used to find candidate schools for the 
evaluation and then the process used to select schools for the final sample. (To protect the schools’ 
anonymity, they are arbitrarily referred to as school A, school B, school C, and so on in fiiis report.) The 
chapter closes with a characterization of the sample schools and the third-grade students in those schools. 

I. Selection Process 

Site selection took place in two stages, as illustrated in Figure 3.1. In the nomination stage, 
ASP staff identified elementary schools that had launched Accelerated Schools within the appropriate 
time fi"ame.^ Nominations were necessary because no standardized documentation that might have 
allowed the researchers to identify Accelerated Schools independently was available. In the filtering 
stage, the researchers eliminated nominees that did not meet the implementation and historical data 
requirements or did not serve the target population. Service to the target population was determined 
based on the following factors associated with school failure: receipt of Title I benefit (which provides 
discretionary fimds to schools to help them serve disadvantaged students), having a high percentage of 
students who are eligible for a fice or reduced-price lunch benefit, and having a high percentage of 
minority students. The nomination and filtering stages were conducted without knowledge of schools’ 
test scores. 



A. The Nomination Stage 

Staff at ASP’s National Center and at the Satellite Centers were asked to identify schools that 
would be appropriate for the evaluation. To tiiis end, the National Center developed guidelines to 
screen potential schools for maturity, that is, degree of experience in using the model. To qualify as a 
mature Accelerated School, a school had to have implemented most of the following components of 
the reform:'* 



^Hopfenberg et al., 1993. 

^Initially, schools had to have launched the reform by the 1993-1994 school year. As sample selection 
progressed, the period during which laimch had to have occurred was lengthened to include 1994-1995. At that 
point, nominees that had laimched in 1994-1995 then entered the filtering stage. 

^The bracketed information, which was not part of the guidelines used to confirm nominees’ eligibiUty, is 
intended to clarify the conq)onents for readers. The rest of the list is quoted verbatim from the National Center’s 
guidelines. 
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Accelerated Schools Evaluation 
Figure 3.1 

Sample Selection Process 
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• Priority setting. The school sets priorities based on taking stock [of the school’s 
strengths and weaknesses] and a shared vision. 

• Governance. Governance structures (the cadres, the steering committee, and SAW 
meetings) are in place and functioning using inquiry, meeting on a consistent 
basis, and using the SAW as a decision-making body and the vision as a decision- 
making filter. A irimimum of one cadre has completed a full cycle of inquiry [the 
process by which school staff examine problems and brainstorm solutions]. 

• Integration of the Three Accelerated Schools Principles and Nine Values. The 
school’s culture and climate have changed in ways that encourage participation, 
risk-taking, experimentation, reflection, and collaboration among and between 
faculty, staff, student, parents, and/or the community. The school culture is one 
that emphasizes equity. 

• Pofwerfiil learning. The school has created powerful learning experiences to 
accelerate the progress of all children. 

• Coaching. An on-site or off-site coach has regularly visited the site to guide 
implementation of the model. 

• The school is implementing powerful learning. 

• Classroom practices have changed in alignment with the philosophy of the 
powerful learning firework, moving the school towards its vision. 

The nomination stage was iterative: As nominees were filtered out of the pool of potential 
candidates, additional nominations of ehgible schools were sought. About 100 schools were 
nominated in all. They represented the majority of schools that had become associated with the project 
within the ^propriate time finme for inclusion in the study. However, the nominators determined that 
some of the schools that they had nominated were not ehgible for participation in the evaluation. For 
example, some were not official ASP schools because they had not participated in training or had not 
remained in contact with ASP staff after training. Others were not yet mature according to the 
guidelines hsted above. These schools were eliminated, leaving approximately 70 ehgible nominees to 
enter the filtering stage. 

B. The Filtering Stage 

The filtering stage was carried out by the researchers independently of ASP to determine 
which nominees would be included in the final sample. In this stage, the researchers determined 
whether the ehgible nominees met implementation requirements,^ served the reform’s target 
population, and met the foUowing research requirements: 



^As discussed in Chapter 4, field research subsequently revealed widespread concision about and uneven 
implementation of powerful learning. During the site selection process, however, a school was considered to have 
implemented tins conqionent as long as the contact person reported that powerful learning was occurring for all 
students. In a departure from the National Center’s guidelines, use of a “coach” was not considered in the filtering 

(continued) 
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• Data. To ensure that any observed changes in achievement were not due to 
changes in the types of tests or scoring used over time, the schools had to be able 
to provide standardized test data that were comparable across all eight years of the 
study period. 

• Launch date. The schools had to have laimched the reform in or prior to the 1994- 
1995 school year. 

• Grade size. For the impact analysis to be able to detect effects on the test score 
measures used, the schools had to serve a minimum of 20 third-grade students in 
each year of the study period. 

• Grades served. The schools had to be elementary schools that served at least the 
first, second, and third grades to ensure a reasonable degree of confidence that 
most students received consistent, substantial exposure to the reform, although 
some of them (in particular, students who transferred to the school fi’om a non- 
Accelerated School) received less exposure. 

• Stability. The school environment and the characteristics of the student population 
had to be stable over the study period to help rule out alternative explanations for 
any changes in achievement observed after implementation of the reform. 

• Magnet schools. The schools could not have a magnet program in place that 
affected the third grade or earher grades. 

AH 70 schools that advanced to the filtering stage met the appHcable implementation and 
research guidelines. The data requirement was the most difficult to meet: Only somewhat more than 
half of the 53 schools contacted for further review could provide the necessary historical data. 
(Seventeen schools, most of them in districts or states in which many mature schools had been already 
selected for the sample, were not contacted for further review.) Other reasons for elimination at this 
stage included not serving students in the first three elementary grades, having laimched the reform 
after the 1994-1995 school year, and serving too few students at risk of educational failure. The 
selection process yielded a final sample of eight schools. 

n. Schools in the Sample 

The eight schools in the evaluation were located in diverse settings across the nation, ranging 
fi’om the bustling metropoHses of New York City and St. Louis, Missouri, to the midsized city of 
Charleston, South Carolina, to a small Missouri town composed of about 10 streets. With the 
exceptions discussed below, all the schools met the implementation and research design requirements 
already outlined. 

All but one school (school B) had at least three years’ experience in using the Accelerated 
Schools model prior to being selected for the sample. It was discovered through the field research 



stage because the necessity of receiving ongoing technical assistance was not enqjhasized early in the reform’s 
development. 
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conducted after sample selection was complete that the staff at school B did not “buy in” to and truly 
participate in the reform until the last year of the follow-up period (follow-up year 5). At that school, 
the model was imposed on the staff over the summer break through an autocratic decision on part of 
the principal, who was not well respected. On returning to school in the fall, teachers were informed of 
the decision and trained in how to implement the model, but there was general dissatisfaction with and 
misunderstanding of the approach. With the arrival of a new principal in follow-up year 5 came a turn 
in the tide. Staff support and buy-in grew as teachers’ and schools’ needs were identified and 
addressed. Another school’s (school C’s) introduction to the model was also not altogether voluntary. 
Specific information on whether this affected buy-in was not available, but it should be kept in mind 
that this school’s implementation of the reform may have been adversely affected and the reform’s 
impacts on student achievement therefore weakened. 

The environment at one school (school E) and the student population at several schools were 
found to be somewhat unstable. At school E, adoption of the Accelerated Schools model coincided 
with the appointment of a new principal. Though reform-minded, the principal did not impose the 
model on the staff; instead, it came as a suggestion fix>m the district. The decision to implement was 
put to a vote, and 100 percent of the faculty agreed. Given the unanimous buy-in and active 
participation on the part of the staff, it can be argued that potential changes in achievement are 
attributable to the Accelerated Schools ^proach, and it was therefore included in the sample. At the 
other schools, there was a shift in ethnic composition of the student population during the study period 
(for details, see the next section). 

One school (school C) provided data for the baseline period and the first four years of the 
follow-up period but did not submit data for follow-up year 5 in time for the school’s data fix>m 
follow-up year 5 to be included in the impact analysis. 

in. Characteristics of the Students 

This section examines how closely the student bodies at the schools in the final sample reflect 
ASP’s target population, that is, students at risk of school failure. Test data and demogr^hic data are 
both good indicators of risk. Although test data are used below to describe the sample schools’ 
average level of achievement prior to implementation of the Accelerated Schools model, they played 
no role in the site selection process.® Moreover, because the quantitative demographic data presented 
here were not available when sites were selected, student characteristics were measured solely on the 
basis of staff reports for the pxuposes of site selection. 

A. Baseline Achievement as Indicator of Risk 

A useful way to get a broad, objective sense of a school’s level of achievement in the baseline 
period is to look at the performance of its students relative to that of other students in their state or in 
the country. Table 3.1 shows the average percentile scores in reading and math of students 



^Selecting schools on the basis of the outcome of interest (test scores) could have biased the study’s inqiact 
findings. 
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Accelerated Schools Evaluation 
Table 3.1 



The Percentile Equivalent of Each School's Mean Baseline Score 
Relative to Its State or National Reference Group 



School 


Reading 


Math 


A 


21 


36 


B 


54 


53 


C 


35 


39 


D 


56 


63 


E 


43 


45 


F 


24 


43 


G 


26 


31 


H 


43 


49 


Average* 


37 


46 



SOURCE: MDRC calculations using student test scores and demographic data provided by each school 

NOTES: The san^le includes all third-grade students tested dxjring the study period at each school 

“The percentile rank for the average school was obtained by (1) converting the percentile score for the average 
student at each school to a Normal Curve Equivalent (NCE), (2) confuting the mean NCE for all schools, and (3) 
converting the mean NCE back to percentiles. This process thus uses the cardinal properties of NCEs for confuting 
means and the ordinal properties of percentiles for reporting findings. 
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in each of the study schools in the three-year baseline period relative to the scores of the state or 
national reference group against which the tests were scored. The bottom row of the table shows that, 
across the eight schools, students with reading and math scores that were close to the sample average 
scored somewhat below their state or national counterparts. In reading, the average student in the 
baseline sample scored at the 37* percentile relative to the state or national reference group, with a 
range fiom the 21®* percentile to the 56*** percentile across schools. In math, the average student in the 
baseline sample scored at the 44*** percentile relative to the state or national reference group, with 
scores ranging fiom the 31®* percentile to the 63*** percentile across schools. 

These comparisons indicate that overall the students in the study sample were at risk of 
educational fiiilure; Seven of the schools had an average score below the 50*** percentile in their state 
or nationally in at least one of the two subjects, and six of them had average scores below the 50*** 
percentile in both subjects. However, there was variation among schools. Students in two of the 
schools (schools B and D) had average scores that fell above the 50*** percentile with respect to the 
reference group in at least one subject. 

B. Demographic Indicators of Baseline Risk 

Demogr^hic data indicative of students’ risk of educational failure include their eligibility for 
a fi:ee or reduced-price lunch benefit, which is used as a proxy for economic disadvantage; and racial 
or ethnic minority status, which has been linked to school performance. According to these indicators, 
about two-thirds of the students in the baseline sample were at risk (see the first two columns of Table 
3.2). Overall, about 64 percent of students in the five schools that provided lunch benefit eligibility 
data for the baseline period were eligible for a firee or reduced-price lunch benefit. In three of these 
schools, a majority of students were eligible. In addition, about 66 percent of students enrolled in the 
sample schools at baseline were nonwhite. In five of the eight schools, a majority of the students were 
nonwhite, and in four of them more than 97 percent of students were nonwhite. 

School H, for which data on lunch benefit eligibility fix)m the baseline period were not 
available, served a relatively low percentage of at-risk students at baseline judging fiom the 
rac^ethnicity data it provided. However, about 62 percent of students in this school were eligible for a 
fioe or reduced-price lunch benefit at the time of the implementation study, which was conducted after 
the follow-up period. At the same point, 49 percent of students in school H qualified for English as a 
Second Language (ESL) services, another risk indicator. 

According to the available quantitative data, school B seems to have had only a small 
percentage of at-risk students in the baseline period. Other indicators, however, suggest that school B 
served a considerable number of at-risk students. According to 1990 census data, which were 
collected approximately two years before this school launched the reform, out of the roughly 8,000 
people living in the vicinity of the small town in which the school was located (which had no other 
elementary schools), more than 1,000 were classified as living below the poverty level. Moreover, 
about one-fliird of people aged 25 or older living in the vicinity of the town had not earned a high 
school diploma or its equivalent, and more than 20 percent had less than a ninth-grade education. 
Finally, the principal of school H reported that some children enrolled in the school lived in severe 
poverty, residing in trailers or shacks without electricity. 
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Accelerated Schools Evaluation 
Table 3.2 



Average Demographic Characteristics Prior to and Following Implementation 
of the Accelerated Schools Model, by School 







Percentage 


Percentage 


Percentage 


Percentage 


Percentage 


School 


Study Period 


with Lunch Benefit 


Nonwhite 


Overage for Grade 


Female 


Tested 


A 


Baseline 


55.6 


99.3 


1.4 


47.2 


89.9 




Follow-up 


80.0 


98.6 


0.5 


55.7 


81.7 


B 


Baseline 


37.0“ 


0.7 


9.8“ 


50.3 


91.3 




Follow-up 


33.8 


0.6 


6.5 


50.3 


92.0 


c 


Baseline 


97.6 


97.6 


25.5 


49.7 


84.2 




Follow-up 


98.9 


97.7 


17.6 


46.6 


68.5 


D 


Baseline 


36.5 


47.8 


N/A 


50.7 


100.0 




Follow-up 


49.7 


60.4 


N/A 


47.7 


98.5 


E 


Baseline 


N/A 


97.4 


21.7 


52.2 


79.0 




Follow-up 


N/A 


97.8 


6.4 


47.1 


83.3 


F 


Baseline 


N/A 


99.0 


3.7 


50.9 


94.6 




Follow-up 


38.0 


99.8 


2.8 


47.3 


87.8 


G 


Baseline 


92.9 


55.8 


12.4 


52.2 


73.9 




Follow-up 


83.2 


73.0 


15.2 


51.0 


69.8 


H 


Baseline 


N/A 


27.2 


N/A 


48.6 


99.4 




Follow-up 


N/A 


40.6 


3.5“ 


44.7 


99.4 


Average 


Baseline 


70.7 


65.6 


12.4 


50.2 


89.0 




Follow-up 


63.9 


71.1 


7.5 


48.8 


85.9 



SOURCE: MDRC calculations using student test scores and demographic data provided by each school. 

NOTES: The sample includes all third-grade students tested dining the study period at each school. 
“More than 25 percent of the data are missing; this number should be interpreted with caution. 
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C. Demoeraphic Shifts over the Study Period 



To determine whether adjustment for demographic changes would be necessary in the impact 
analysis and whether changes in test scores were just an artifact of a changing student population, the 
stabiUty of student characteristics was also examined. The population of students enrolled in a school 
may change over time for a variety of reasons. For instance, the residents in the school’s catchment 
area (the area it serves) may change, or another school in the neighborhood may open or close. 
General demographic data on the eight schools in the study sample and the percentage of students 
tested in the baseline and follow-up periods are presented in the last three colimms of Table 3.2 (for 
the same data broken down by year and school, see Appendix A). 

The proportion of students who were eUgible for a free or reduced-price lunch benefit in the 
study schools as a group rose somewhat from baseline to follow-up (from 64 percent to 71 percent). In 
schools A and D, there were larger increases over the study period in the percentage of students who 
were ehgible for a free or reduced-price lunch benefit. 

Overall, there was a sHght increase from baseline to follow-up in the percentage of nonwhite 
students (finm 66 percent to 71 percent). Five schools remained homogenous with respect to 
race/ethnicity over the whole study period. In the other three schools, there was an increase in the 
percentage of nonwhite students and a corresponding decrease in the percentage of white students 
over the study period. In school G, the decrease was sharp following enactment of a local law 
requiring integration, which resulted in the busing of black students to the school. Over the study 
period, the student population in school G shifted finm being predominately white to predominately 
black. In school F, the percentage of black students increased while the percentage of Hispanic 
students decreased. 

Another measure of the stability of student characteristics is the percentage of students over 
the study period who were overage for their grade. The average age of the students in the sample (all 
of whom were third-graders), computed relative to a test date of April 15, was about 9 years in each 
school year.’ For the analysis, students aged 10 years or older are considered overage for the third 
grade.^ Though the average age in the study schools remained essentially the same from baseline to 
follow-up, the percentage of overage students decreased somewhat (fix>m 12 percent to 7 percent). 

In each school and each year of the study period, about half of the students in the sample were 
female. The only exception was school A, where there was an increase in the percentage of girls fix>m 
baseline to follow-up. 

Only students for whom test data were available are included in the impact analyses presented 
in Ch^ter 5. A particular student’s test data might have been missing for a variety of reasons, for 
instance, because the student transferred out of the school prior to testing or was excluded from testing 
owing to a language barrier. To determine what percentage of students in the study sample were 
tested, the number of students for whom test data were available in each study year was compared 



’All the schools administered tests in the spring semester of each year in the study period. 

®Age data for the four students in the sairqjle who were recorded as being under 7 years old or 10 years or older 
were coded as missing owing to the questionable reliability of these data. 
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with a roster of all students enrolled in the third grade in that year.^ The overall percentage of students 
tested ditting the baseline period was fairly similar to that tested diuing the follow-up period: 89 
percent and 86 percent, respectively. This percentage was also similar fiom school to school, with the 
notable exceptions of schools A, C, and F. 

At school A, the percentage of third-graders who were tested declined in the last year of the 
baseline period, decreasing fiom an average of 94 percent in baseline years 1 and 2 to an average of 82 
percent in baseline year 3 because some special-education students were excluded fiom testing be ginning 
in baseline year 3. Had these students been tested, the percentages of students tested in the baseline and 
follow-up periods would have been comparable. At school C, the percentage of third-graders tested 
decreased fiom 84 percent at baseline to 69 percent at follow-up. A contact fiom the school sug^sted 
that the difference there might have been the re^t of an increase in the percentage of special-education 
students excluded fiom testing (as it was at school A). However, this hypothesis could not be 
investigated because special-education data were not available for this school. 

At school F, the percentage of students tested decreased from 95 percent at baseline to 88 percent 
at foUow-iqj because of a marked drop in the percentage of students tested in foUow-iqj year 5 (to 75 
percent). In foUow-iqj year 5, a larger percentage of non-EngLsh-spealdng students were exempted fiom 
testing than had been in earlier years, which may have increased the average test score in that year 
relative to those in earlier years. However, data for follow-iq) year 5 also included students who attended 
another school housed at the same location who were not exposed to the Accelerated Schools ^roach. 
Including these students, who were considered to have behavior problems, may have concomitantly 
decreased the school’s average test score in foilow-iqj year 5. 

Based on an analysis of the size and composition of the student sample over the study period, 
it was concluded that changes in these variables could not accoimt for any observed impacts of the 
reform.'® Though the minor demographic shifts revealed by this analysis were controlled for in the 
estimation of impacts of the Accelerated Schools model on test scores, the impacts aggregated across 
the study schools are essentially the same whether they are adjusted for demographic changes or not 
(for adjusted impacts, see Table 5.3; for the corresponding unadjusted impacts, see Appendix A). 



rv. Summary 

The sample of eight schools examined in the present study of Accelerated Schools meets the 
requirements necessary to allow inferences about the effectiveness of the Accelerated Schools model 
in improving student achievement. The schools were chosen through an impartial process fiom a 
group of candidates identified as Accelerated Schools. Overall, the implementation of the reform in 
the schools in the final sample was typical of that in all schools that launched the reform early in its 
evolution; that is, these schools employed the components of the model, particularly those related to 
governance and decision-making, that were emphasized at the time. 



’a con^lete roster was not available for school C in baseline year 2 or for School E in follow-up year 1. As a 
result, data for these schools in these years are not reflected in the average percentage of students tested. 

'°See Bloom, 2001. 
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Most of the students served by the study schools were at risk of school failure and 
therefore in the population targeted by the reform. On average, these students scored below their 
state or national counterparts on standardized math and reading tests. Throughout the study 
period, two-thirds of them were at risk based on available demographic indicators. In the case of 
schools that could not be classified in this way for lack of the necessary data, other data — 
including staff reports, census data, and ESL data — suggest that these schools also had a 
substantial proportion of at-risk students. 

With minor deviations, all the study schools met the requirements of the research design. The 
schools were able to provide a sufficient amount of historical data for the impact analysis to be able to 
detect any change in students’ average math or reading test scores, and the environments and third- 
grade studait populations were sufficiently stable over the study period for any observed changes in 
test scores to be attributable to the Accelerated Schools model. 
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Chapter 4 



Implementation of the Accelerated Schools Reform 
in the Study Schools 



An important component of the present evaluation of the Accelerated Schools model was 
the implementation study that was conducted to document the model’s implementation in the 
eight sample schools. This chapter describes the methodology used in the implementation study 
and presents the study’s findings (for the protocols used to collect the qualitative data presented 
in this chapter, see Appendix B). 

The primary goal of the chapter is to describe how elements of the Accelerated Schools 
approach were implemented in the schools’ routine practice. To that end, illustrative examples of 
the approach in action as reported by staff are presented. The chapter also highlights implementa- 
tion trends observed across the sample schools that may help explain the impacts presented in 
Chapter 5. Lastly, the chapter presents information verifi^ng that the sample schools met the ma- 
turity criteria (with respect to implementation of the Accelerated Schools approach) defined by 
staff at ASP ’ s National Center. ' 

I. Methodology of the Implementation Study 

The implementation study was conducted during the 1998-1999 academic year to obtain a 
“snapshot” view of the sample schools with respect to governance and decision-making, school 
culture, and curriculum and instruction.^ The study also included an examination of the types of 
technical assistance that schools received in implementing the Accelerated Schools approach. 

Case study profiles covering the study’s four focus areas (governance and decision- 
making, school culture, curriculum and instruction, and technical assistance) were compiled for 
each school during a two-day site visit. Each visit included (1) interviews with school administra- 
tors, lead staff for curriculum planning, and the Accelerated Schools coach (where available); (2) 
focus groups with teachers and other members of the school community who were actively in- 
volved in school decision-making processes; (3) observations of third-grade classrooms; and (4) 
document review.^ 

Using the case study profiles, a content analysis approach was then used to create text- 
based data tables, which served as tools to examine trends in implementation of the Accelerated 
Schools approach. The data required for the analysis focused on school operations and school 
conditions (in the four focus areas) that 

• were being implemented or existed school-wide within grade levels (with spe- 
cial attention to primary grades); 



'For detailed information on the criteria used to assess the schools’ implementation maturity, see Chapter 3. 

^Six schools were visited during the 1998-1999 school year; two schools were visited during the 1999-2000 
school year. 

^The documents reviewed typically included (where available) the school’s vision statement, taking-stock re- 
ports, school improvement plans, and cadre membership lists and recent minutes. 
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• precise and readily recognizable representations of the philosophy and pre- 
scribed processes of the Accelerated Schools approach; and 

® were documented in multiple sites, providing a standard data set from which 
comparisons and trends within the study sample could be drawn. 

Table 4.1 provides an overview of the four areas of school functioning that were targeted 
for examination in this study, along with the processes or model components associated with each 
one.'* The table also presents the focus of the goals and the focus of documentation in each area. 

II. School Governance and Decision-Making 

In examining school governance and decision-making processes, the analysis focused on 
three things: (1) schools’ implementation of the Accelerated Schools governance structures, (2) 
schools’ usage of the Accelerated Schools processes of taking stock and inquiry, and (3) school 
staff perceptions of how the Accelerated Schools approach decision-making processes influenced 
the achievement of improvement goals. As described below through staff comments and illustrative 
examples, implementation of the Accelerated Schools governance structure was well integrated into 
the routine practice of school life, affecting how the sample schools’ improvement priorities are 
now identified and addressed. This is an important finding because such institutionalization was 
consistent with meeting the maturity criteria for implementation of ASP governance. 

The examples presented below are also important because they provide evidence that the 
ideas generated through ASP governance structures have shaped school-wide operations in a 
number of ways. Trends in this regard indicate that during the initial years of implementing the 
Accelerated Schools approach, school staff focused on climate issues such as student discipline, 
school facilities, and parental involvement. In the later years of implementation (that is, in fol- 
low-up year 4 and beyond), staff decisions that were stimulated through the use of ASP govern- 
ance structures tended to focus on curriculum and instruction. Implications of this trend are ex- 
amined in Chapter 5, which presents impact findings from the evaluation. 

This chapter does not identify any school by name, in order to protect the confidentiality 
of individual informants whose comments might link them to a specific school. Further, the find- 
ings do not indicate a systematic pattern of stronger implementation at some schools than at oth- 
ers; the sample schools varied in their areas of strength. The main pattern across the schools is 
the initial focus on school governance and culture, followed by growing attention to issues of 
curriculum and instruction. 

• The Accelerated Schools approach decision-making structures were cen- 
tral to governance operations in the sample schools. 

Staff reported and provided documentation to demonstrate that a decision-making process 
was in place and that it reflected the Accelerated Schools governance process and structures. In 



^Definitions used for the areas of school functioning were generated by the evaluation team based on a review of 
school reform literature and are not specific to the Accelerated Schools approach. These definitions are intended to 
characterize how these areas of school functioning are described in the broader education research field. 
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Accelerated Schools Evaluation 
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^The alignment of school fimctioning areas to specific structural and philosophical elements of the Accelerated Schools approach was determined by the evaluators for 
ease of presentation and analysis. In practice, developers of the Accelerated Schools approach espouse that elements of the model overlap, affecting multiple areas of school func- 
tioning (and each other) in a dynamic and reiterative fashion. 



other words, the sample schools regularly implemented a process to identify priority needs (tak- 
ing stock and vision setting); they created cadres to investigate, or conduct “inquiry” on, targeted 
issues and problem areas; and steering committee members reviewed cadre proposals, which (if 
approved) would go to the school-as-a-whole (SAW) for input and final decisions. 

The use of cadres, the steering committee, and the SAW in formulating school change 
ideas and making decisions was consistent across the sample schools. The cadres that were estab- 
lished using the Accelerated School approach tended to be viewed as instrumental and driving 
forces in school-level planning. During the time of the site visit,^ all but one school had cadres in 
place that met an average of once a month; the number of cadres during any given school year 
ranged fi’om three to six. Each school also had a steering committee, made up of lead representa- 
tives fi’om the cadres along with school administrators, that convened monthly to get updated re- 
ports on each cadre’s work, review cadre proposals, and discuss overall issues affecting the 
school. Similarly, the sample schools reported that they held monthly SAW meetings at which 
the broader school community could provide feedback and participate in decisions about pro- 
posed strategies. 

It was routine for teachers and school administrators to participate actively as members of 
the cadres, the steering committee, and the SAW. Participation among other school community 
members was less consistent. In five of the sample schools, noncertified school staff fi’equently 
participated in cadre, steering committee, and SAW activities. One school reported that its gov- 
ernance meetings regularly included a central office district administrator and a community vol- 
unteer. It was rare, however, for students and parents to participate regularly in Accelerated 
Schools governance structures.® 

Staff in nearly all the sample schools cited lack of sufficient time as a key challenge to 
sustaining implementation of the Accelerated Schools approach to governance and decision- 
making. Nonetheless, meetings of ASP governance structures have become a fixed element of 
their school-year calendars, and some schools have been able to build meeting time into the regu- 
lar school day. For example, one school extends the length of its school day and thus banks the 
equivalent of five days over the course of the school year to make time for monthly cadre plan- 
ning meetings. Another school releases its students two hours early every Wednesday throughout 
the school year, in order to conduct staff planning meetings. (The cadres meet every first 
Wednesday of the month, the steering committee meets every second Wednesday, and the SAW 
convenes every third Wednesday; the fourth Wednesday is reserved for grade-level meetings.) 
Similarly, staff at a third school have been creative (and persistent) in their efforts to secure the 
time needed for governance meetings. A site visitor fi’om the evaluation team described the evo- 
lution of this school’s strategy: “They initially tried to create cadre meeting time during class 
time by placing other personnel in classrooms [to release teachers], but found this was not feasi- 
ble. Ultimately, the school got a waiver fi’om the state to begin school five minutes earlier and 
end after lunch one day a week. The time they freed up was used for cadre meetings.” 



^One sanqile school did not have cadres in place because its focus was on selecting a new principal for the up- 
coming school year, which therefore deferred other staff activities. 

*Most schools reported that parent participation was encouraged but difficult to achieve consistently. 
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o The sample schools appear to have institutionalized the use of taking- 
stock and inquiry processes to identify and examine their improvement 
goals. 

The initial taking-stock process was viewed as a major undertaking by most of the sample 
schools. As described in Chapter 2, taking stock is a reflective process that helps the school 
community to assess its strengths and weaknesses, to establish or reaffirm a shared vision of 
learning for the school, and to identify goals to achieve that vision. 

Several sample schools have instituted an ongoing taking-stock process to allow for re- 
flective management of their improvement goals. In particular, one school has made taking stock 
an annual process for shaping the agenda for change. Before the start of each school year, staff 
are convened for two days for the purpose of identifying improvement goals for the coming year. 
Teachers work in grade-level teams to review the previous year’s test scores and the results of 
teacher, student, and parent surveys conducted each spring. Target areas are identified and be- 
come the focus of cadre work for the school year. School staff describe this process as “mini tak- 
ing stock” and report that it helps to keep cadre work relevant: “We don’t hold on to a stagnant 
cadre.” Similarly, staff at another school use an end-of-year teacher survey to guide their plan- 
ning. The survey asks teachers to reflect on what went well during the school year and what 
could be improved. Results of the survey are reviewed during the summer and are used to estab- 
lish improvement priorities for the coming year. At a third school, an annual taking-stock process 
is used to refine the school’s vision and reaffirm or identify new cadre focal areas: 

We have a vision statement and it isn’t set so that it stays. It changes as we think it 
needs to change. ... At the end of each year, the school-as-a-whole reflects on the 
past year and sets the cadres for the following year. . . . We talk about what 
worked and what didn’t work, and then that kind of sets the agenda for what we 
want to think about for the next year and that generates the cadres. 

School staff also reported that they use an ongoing inquiry process — albeit modified, in 
many instances, to accommodate time constraints and local regulations. As recommended in the 
Accelerated Schools Resource Guided staff were likely to use the inquiry process when they had to 
make determinations about adopting or modifying activities that affected school-wide operations. 
Decisions affecting a specific grade or other smaller portion of the school community continued to 
be made through more traditional mechanisms, such as grade-level meetings or an administrative 
team. Often in such cases, a cadre would not be assigned to investigate, nor input solicited firom the 
SAW, before finalizing a decision to implement a particular strategy or change. The issue of time 
was the primary reason cited for not fully using the inquiry process to guide all school-level deci- 
sions, as reflected in the following comments fix)m staff representing various schools: 

We went through the [inquiry] process the first time exactly as prescribed. It took 
a year and a half. [Since] then we accelerated the acceleration part — any shortcut 
we can find that doesn’t take away fix)m the process. 

We never used the inquiry process all the way. A lot of what they said in the [Accel- 
erated Schools Resource Guide] was too laborious, too much paperwork . . . when 



^Hopfenberg et al, 1993. 
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[oiir school] went to modified calendar we used the [inquiry] process thoroughly. 

We spent a year researching the issue before deciding to put it into place. 

We stopped doing every step [of inquiry]. We had to tailor-make it. . . . We be- 
lieve inquiry is a fi"amework — we don’t need to do it the exact way. That’s why 
some schools have difficulty with the program, because they want to do every step 
exactly the way the {Accelerated Schools Resource Guide] says. For instance, with 
inquiry, you’re supposed to look at all the possible ways to achieve a goal. We 
identify the ways we want to achieve the goal without discussing every possible 
way to make it happen. 

Staff at a school that used inquiry to inform the decision to implement a modified- 
calendar school year provided one example of the inquiry process in action. This school’s deci- 
sion was based on a two-year research and consensus-building process that utilized inquiry tech- 
niques to inform work of the cadres, the steering committee, and the SAW. The initial idea to 
investigate a modified calendar arose in response to students’ comments on a survey, which indi- 
cated that they wanted to be at the school more. As described by the school’s Accelerated 
Schools coach: 

It happened gradually. Om taking-stock student surveys showed that kids wanted 
more time in school. One student even said he wanted [the school] to become a 
boarding school. We felt we needed to provide more time for kids to learn. This 
gave faculty the impetus for thinking about ways of doing that, which eventually led 
to the modified calendar. In 1995, a group of teachers were at a conference and 
heard a presentation on year-round schooling and that was the first time someone 
said out loud, “Why don’t we t hin k about a year-round school?” The idea was pre- 
sented to the school-as-a-whole to develop a cadre that would focus on studying the 
feasibility of year-round. In 1996 we did an in-house study with pre/post-testing of 
students. We tested students at the end of the school year and again in the fall and 
students’ scores had dropped. In 1997 we developed a modified-calendar cadre to 
research the modified schedules. Faculty visited schools in [two other states]. We 
did surveys of the staff and had a principal fi-om a modified-calendar school visit 
[our school]. In [Spring] 1997 the cadre presented [a plan] to the school-as-a-whole 
and they voted to make a request to the school board to begin implementation in Fall 
1998. After we got approval fi-om the board we presented the idea to parents — we 
did presentations in housing developments and chinches. 

In 1998, this school became the first school in its county and one of 10 schools in its state 
to implement the modified school calendar. The school year begins in July and operates on nine- 
week semester cycles separated by three-week intersessions. The first week of intersession is a 
program that provides remediation to targeted students who are lagging behind academically; the 
second week’s program provides a theme-based enrichment curriculum targeted at all students; 
and the third week is vacation. 

9 Ideas generated through the decision-making structures of the Acceler- 
ated Schools approach have shaped school-wide operations in ways that 
are helping the sample schools achieve their improvement goals. 
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School staff readily cited outcomes from using the Accelerated Schools approach that 
they felt significantly enhanced the school’s learning environment, and cadre work served as the 
impetus for many of these changes. Planning based on the processes and structures of this ap- 
proach has led to school changes ranging from new playground designs, to student attendance 
monitoring policies, to the length of the school year. The pattern of decision-making fostered by 
the Accelerated Schools approach indicates an initial focus on issues of student disciphne, school 
facilities, and parental involvement before moving on to specific content areas of curriculum and 
instruction. As teachers during one focus group described it: “Initially we worked on a lot of pet 
peeves and pet projects.” For example, at one school, teachers reported that their facilities cadre 
made great strides. The cadre members prompted the school to send the district office a list of 
repairs that were needed, and they got parents involved in the lobbying effort. They also invited 
television cameras to the school to videotape trouble spots (such as rusted sinks). Using the me- 
dia turned out to be an effective strategy — the sinks were fixed the following year. 

Staff at another school credited the Accelerated Schools approach with changing teacher 
compensation policies. Prior to implementing this approach, grade-level chairpersons at the 
school were given a stipend to compensate them for the additional time they spent in fulfilling 
administrative duties for their departments. With implementation of the Accelerated Schools ap- 
proach, however, all teachers in the building began taking on leadership and planning roles. The 
SAW agreed to negotiate with the local teacher’s union and was successful in having the stipend 
funds divided among all faculty members. Although the amount of stipend per teacher was not 
significant, this action was cited by staff as symbolizing the school’s commitment to changing 
the decision-making dynamics among staff and to supporting a culture of shared responsibility 
for students. 

During the early years of Accelerated Schools implementation at a third school, a cadre 
was formed that focused on student discipline. This cadre helped the school to identify and insti- 
tute a nationally known student discipline program that led to the creation of school-wide rules of 
conduct and included teacher training. In addition, the cadre generated ideas to modify activities 
in common areas of the school, like the playgroimd and the cafeteria, because a lot of student 
confrontations were foimd to start in such areas. Based on this cadre’s proposal, the school re- 
named its cafeteria after the school’s mascot and piped in music during the lunch hour in order to 
calm students’ behavior. The school also replaced the long cafeteria tables with small, round ta- 
bles and encomaged teachers to eat lunch with the students (a practice the researchers observed 
during the site visit). Staff at this school reported that, in combination, ideas generated from the 
cadre worked well in quelling student discipline problems: “Oiu" students’ behavior turned 
around immediately.” 

The focus of work that has been spearheaded by Accelerated Schools governance proc- 
esses and structures has, over the longer term, shifted toward specific content areas of curriculum 
and instruction. The statements of one school administrator captured the most common reason 
cited for this pattern of evolution: 

We’re starting to [focus] more [on] curriculum and instruction which was an intent 
five or six years ago, but there were so many other areas that needed to be addressed 
that were strongly in om faces that we had to deal with. Our facility, the building, 
was horrendous; it was just terrible. A lot of those things have been addressed and 
have improved so much that now we’re at the point where we can start focusing on 
curriculum and instruction and that’s why we have the new cadre areas. 
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During the time of the site visits to the sample schools — at least five years after the 
launch of the Accelerated Schools reform in these sites — the majority of cadres that were in 
place focused on instructional content areas such as reading, writing, and mathematics. Section 
IV of this chapter describes in detail the ways in which curriculum and instruction have changed 
at the sample schools since they began implementing the Accelerated Schools approach. 

III. School Culture 

Perceptions of school staff — primarily teachers’ — were used to understand the nature 
of relationships and the work culture operating in the sample schools. The examination of school 
culture focused on teachers’ perceptions centering on three issues: (1) coherence and unity of 
purpose in the school’s educational program, (2) trust and collegial support, and (3) teacher input 
into school-level decisions. 

As the comments below indicate, the major finding about school culture is that the Accel- 
erated Schools decision-making structures and processes have been effective in creating and/or 
nurturing a school culture that reflects the principles and values that undergird this approach. The 
trend across schools was that teachers perceived their work environment as encouraging broad- 
based participation and equity in decision-making and as promoting an atmosphere of collabora- 
tion, trust, and experimentation. Such perceptions are consistent with meeting the maturity crite- 
ria for ASP implementation. 

Particularly noteworthy is that teachers felt that the Accelerated Schools approach fos- 
tered unity by strengthening connections among various program initiatives that were operating 
in their school. This is important because past research has indicated that perceived “coherence” 
in academic progr ammin g planning is critical to the successful implementation (and thus the out- 
comes) of school change efforts.* 

• Teachers reported strong feelings of trust and support from other teach- 
ers and administrators at their school. 

At one school, teachers credited the Accelerated Schools decision-making structures with 
changing collegial dynamics of the staff: “It became more of a team effort rather than going into 
your classroom, shutting the door, and starting to work without being connected to the whole 
[educational] program.” This sentiment appeared to be especially relevant for fine arts, physical 
education, and special resource teachers, who reported that their cadre involvement enabled them 
to interact with other teachers on curriculum planning in a way that was not possible through the 
traditional structure of grade-level meetings. A reading teacher explained the impact of the Ac- 
celerated Schools approach this way: 

I was a remedial reading teacher . . . more or less . . . and many times what [I was] 
doing was not really connected with what the classroom teacher [was] doing. ... 

Once we became an Accelerated School, I was able to put my ideas out . . . [and] 
my responsibihty as a remedial teacher changed somewhat. I would go into the 
classroom and set up an early literacy program that really bonded [the remedial 
teachers] with the classroom teachers. ... It was not just first and second grade; 
we worked with the whole staff and om emphasis [as a school] is now on literacy. 



*King and Newmann, 1999. 
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... I think being able to be a decision maker in the Accelerated Schools process 
helped that along. 

Comments from a teacher focus group at another school reflected similar sentiments: 

A strength of the [Accelerated Schools] process is that you can have differing 
viewpoints and people don’t get ugly about it. The faculty is congenial and re- 
spectful of each other. . . . It’s not like this in other schools [I’ve worked in]. In 
other schools you see teachers in one grade level backstabbing other teachers. 

Here it feels like a family. . . . We have unity because we plan together. We 
wouldn’t have that if we didn’t have Accelerated Schools. 

Staff at another school described their work environment prior to using the Accelerated 
Schools approach as “isolated” with “little positive rapport, communication, or collaboration 
among teachers.” With implementation of the approach, teachers reported that they “work to- 
gether and collaborate within their grade as well as across grades.” An administrator at this 
school described the impact this way: 

[The Accelerated Schools approach helps us see that] it’s important to be reflec- 
tive, collaborative, to do what works. When I came everyone did their own thing. 

It was secretive. . . . Now, there’s feedback, collaboration, and planning. 

Teachers at another school echoed the belief that the Accelerated Schools approach nur- 
tured collegiality: 

Before Accelerated Schools, when we had a problem and people didn’t agree [on 
how to solve it], we didn’t have a process in place to talk about how you can dis- 
agree, but still work something out. Before [teachers] wouldn’t share things with 
each other imless you were part of this group of teachers or that group, but nothing 
was open to everyone. . . . There is tremendous interaction among the teachers. 

It’s not threatening to ask for help. It was not always this way. At first, people 
were hesitant to join the cadres, but it has helped our socialization. A lot of 
schools you go into don’t have that trust. 

• Teachers felt that implementing the Accelerated Schools approach had 
enhanced their input into decision-making, bringing more balance to the 
exchange of ideas between administrators and other school staff. 

Generally speaking, teachers in the sample schools felt empowered and tended to be satis- 
fied with their level of decision-making input and communication mechanisms. In fact, school staff 
typically reported that the Accelerated Schools approach provided, for the first time, sufficient tools 
to make their input into decisions equitable. Nearly all schools in the sample reported that previous 
attempts to establish inclusive decision-making had fallen short. Below are sample comments from 
administrators and teachers who shared the view that decision-making had been improved: 

We had school-based management programs before . . . , but they never gave us a 
process — a how to get there. [Accelerated Schools] gave us a process. 
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The Accelerated Schools program has given us a structure. We work as a team. 

Everything we do comes out of the [Accelerated Schools governance] proc- 
ess. . . . People can trust the process will give them an opportumty to have input. 

In the past strong personalities have been dominant in decision-making. Now it’s 
more balanced. 

The first thing that came out of the cadres was their ability to get everyone in- 
volved. There’s been no school-based program to ever come through here that 
ever gave this type of opportunity to participate. 

Staff here are really involved in all aspects of decision-making, adopting curricu- 
lum, [and input into] anything that is really going to directly affect us. We are a 
part of [decision-making] whether we are approving [something] or whether we 
are designing the change, or finding out if we need to make the change. Teachers 
are really a part of it. . . . We are really into our decision-making, and our opinions 
and ideas are valued. 

All ideas here are equal. No matter if you have 20 years of experience or one year 
of teaching. [The Accelerated Schools approach] made it that everybody can have 
a voice, no matter your comfort level. If you’re not comfortable talking in a 
whole-staff meeting, you might feel comfortable speaking out at a cadre meeting. 

We don’t have a dictator here . . . [the principal] is willing to let the rest of us 
have say and power. . . . We’re all kind of equal sitting at this table and that makes 
a big difference. And [the principal’s] not just playing the role and saying, “Yeah, 
we can all do that,” and have our opinions, but then goes back to her office and 
does what she wants to do anyway. That’s one of the reasons that I’m here. 

[The] Accelerated Schools [approach] did emphasize that everyone be invited to 
the table. We have a custodian here that said this was the first time anyone in the 
school had asked for his opinion about anything. It’s a very moving process. 

At one school, the Accelerated Schools process was credited with helping to develop sys- 
tematic staff communications. Staff post an “Accelerated Schools Bulletin Board” that displays a 
copy of the school’s vision statement; minutes of cadre, steering committee, and SAW meetings; 
and a list of systematic changes that have been implemented since the launch of the reform. The 
bulletin board is located in a well-traveled hallway of the building and is used both to inform and to 
provide feedback, allowing teachers to see “at a glance” what they have accomplished through the 
Accelerated Schools process. Procedures for regularly posting or otherwise distributing minutes of 
cadre and steering committee meetings are also in place at four other schools in the sample. 
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• Teachers expressed a sense of cohesion among elements of their educa- 
tional programs, which they tended to attribute in large part to the coa- 
lescing nature of the Accelerated Schools governance structure. 

The Accelerated Schools approach was frequently credited with providing mechanisms 
that promote an awareness of and coherence among the various school improvement strategies 
that teachers found themselves implementing at the sample schools. For example, teachers at one 
school reported: 

We do have quite a few initiatives [here], but in my opinion it is because of the 
principles of Accelerated Schools and the governance structure that we have been 
able to put all of the things that we do under [one] umbrella. . . . There may be 
times when you hear, “Oh boy, here comes something else,” [because] we do have 
Coalition of Essential Schools, Accelerated Schools, the critical friends group, 
and Caring Communities, but all of these efforts are working toward the same 
end. . . . [Through] the governance structure of the school [teachers] can come to 
see that everything fits under the umbrella of Accelerated Schools. 

Teachers at another school reported similar benefits from using the Accelerated Schools 
approach, especially in terms of building on strengths, which required breaking the habit of con- 
stantly replacing old initiatives with new ones. As one teacher explained: i 

We traditionally have tried things and if they didn’t work, we’d throw it out and 
get something different. We’ve learned that it’s not the program that you should 
go out and buy, but the framework for learning that you should adopt and then 
build on so you’re not always starting something totally new. 

The principal at a third school reported that the Accelerated Schools approach had the same ef- 
fect on her school: 

Before there was a lot of energy, but it was going every different way. We never 
did the same program two years in a row. [The Accelerated Schools approach] 
gives us more time to give thought to a program and make it better. This was the 
biggest change Accelerated Schools has brought along with a guarantee from 
[teachers] that they will follow through because they had agreed to do it. 

At a fourth school, the Accelerated Schools governance structures were viewed as central 
to the successful implementation of two critical events that required building staff consensus 
around educational goals. One was the integration into the school of a bilingual program that 
served 200 students and included the addition of 1 1 teachers.^ The second was the completion of 
the school’s annual comprehensive educational plan, which is mandated by the district for school 
improvement planning. The integration of the bilingual program into the everyday functioning of 



’a portion of this school’s facility was used to house a separately operated bilingual program At the start of the 
1997-1998 school year, the bilingual program officially came under the main school’s adininistration, resulting in the 
addition of 200 students and 1 1 teachers. 
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the main school was facilitated by engaging bilingual faculty in cadre work and as representatives 
on the steering committee. The director of the bilingual program reported: 

If it were not for Accelerated Schools, we probably would not have had as much 
contact with other teachers in the building. Now, we plan with regular classroom 
teachers on aspects of the curriculum that affect the entire school. This planning 
has made a difference in the curricula used in the bilingual program as well. 

In a focus group, teachers described their use of cadres to manage the task of developing this 
educational plan: 

We found that the Accelerated Schools process gave us a good mechanism for 
getting the plan done. We used the ideas from the cadres to create the [plan]. We 
did this through surveys — we looked at a list of all the activities in the building, 
and each cadre had to report back on how its members felt about the activities, 
what they liked and didn’t like and how they would improve it. We started meet- 
ings on the [plan] in February — we met 10 Fridays in a row from 3 to 6 
[o’clock]. Normally, [such a plan] would have been written by [our] administra- 
tive team. By using the cadres, we had so much input and everyone feels very 
proud of the hard work that went into this document. It was a good process be- 
cause the cadres provided a safe forum for people to have a voice and to pass on 
their comments, good and bad, in an anonymous way. 

IV. Curriculum and Instruction 

The study’s documentation and analysis of sample school practices in curriculum and in- 
struction focused on reported applications of the powerful learning framework. As the com- 
ments below indicate, a key finding in this regard is that school staff tended to report difficulties 
in understanding (and thus applying) the powerful learning framework in everyday instructional 
practice. Consequently, the sample schools varied considerably in their choice of curricular pro- 
grams and their emphasis on pedagogical approaches. This inhibits the ability to link these find- 
ings with findings of the impact analysis, because schools cannot be grouped on the basis of 
categories of instructional practice or the strength of their implementation of the powerful learn- 
ing framework. 

It should be noted that school staff did perceive that the Accelerated Schools approach 
had some direct influence on operations specific to curriculum and instruction, centering in par- 
ticular on increased curriculum alignment across classrooms and grade levels and increased 
teacher influence on curriculum planning. The illustrative examples below suggest that the sam- 
ple schools often turned to the Accelerated Schools governance structure (and not to the powerful 
learning framework per se) to reach decisions about curriculum and instruction. This understand- 
ing provides further insight into the process of reaching maturity in the Accelerated Schools ap- 
proach. In particular, it highlights the amount of time needed in implementing the approach for 
schools to begin to change what happens in the classroom. 



'“Given the brief duration of the sample school site visits, the time available for classroom observations was lim- 
ited. It was thus infeasible to objectively assess the inqjlementation of powerful learning concepts at the sanple 
schools, and so this analysis relies on staff reports. 
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9 School staff typically expressed difficulties in relating or applying the Ac- 
celerated Schools approach to powerful learning in their curriculum 
planning. 

As described in Chapter 2, the powerful learning concept has historically been the least 
well-defined component of the Accelerated Schools approach.’’ Not surprisingly, then, staff 
across the sample schools had varying perceptions of what constitutes powerful learning and 
generally had only a vague imderstanding of this concept. 

For example, staff at one school reported that they faced fimstrations in trying to articulate 
the practical aims of powerful learning; 

[Powerful learning] was a continuous problem that persisted through the first year 
of implementation. Few [teachers], if any, felt they ever got a handle on it. The 
difficult thing was in figuring out what it was and how to leam to do it. . . . After 
a session on powerful learning at one of the SAW meetings, what they have gath- 
ered fi'om the [Accelerated School coaches] was mainly that it was learning which 
affected one at an emotional level. That is, it was something that as an adult, 
someone could recollect as a meaningful experience in their lives, powerful 
enough not to have forgotten. ... So befuddled were teachers that they requested 
that an instructor come fi-om [the ASP National Center at] Stanford to conduct a 
special workshop for them on the subject. After the Stanford instructor left . . . 
[teachers] essentially felt they hadn’t received clear messages on what this kind of 
learning was . . . and continued to question their own understanding of the term 
and how to make it work in their classrooms. 

The principal of another school described its interpretation of the powerful learning concept: 

We had staff development on powerful learning. I had a problem with the term. I 
see it as looking at best practices. You look at multiple learning needs, that’s how 
our school took it. We interpret powerful learning to be the best teaching practices 
for your students. The [Accelerated Schools Project] wants you to look at what 
you’re teaching, wants [you] to use more thematic units and open classrooms. 

You can see how our school has interpreted powerful learning. We’re using teach- 
ers’ expertise to determine what is good for our kids. 



Similarly, staff at a third school reported that they were not satisfied with the information 
they received on how to implement powerful learning. According to the principal there, the im- 
pression that staff received fi'om trainers and coaches of the model was that implementation of 
the Accelerated Schools governance structures would bring the school closer to being able to im- 
plement powerful learning. The principal felt that the application of powerful learning was made 
difficult because the concept was described in an ethereal rather than explicit way. 



"Since the time that the study schools launched the Accelerated Schools reform, ASP has refined the powerful 
learning framework to provide more concrete guidance on how to change instruction, and it has encouraged schools 
to begin curriculum and instruction reforms earlier in the process. 
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Staff at a fourth school also found fault with the Accelerated Schools approach for its 
lack of clarity in guiding the implementation of powerful learning. Administrators at this school 
who attended conferences sponsored by ASP stated that they “were not impressed with the pres- 
entation of powerful learning — it was more show than depth.” According to the school’s princi- 
pal: “The original [Accelerated Schools] training had very little focus on powerful learning, 
which was a big mistake. Training focused more on the inquiry process and organizational struc- 
ture [of the Accelerated Schools approach].” 

One school in the sample represents an exception in this regard, and staff reported that 
they were using the powerful learning framework to guide curriculum development. Within the 
school, these teachers developed a powerful learning lab whose participants meet every three 
weeks throughout the year (including the summer) to discuss and design unit lessons that include 
five components of powerful learning. Teachers pilot-test the lessons in their classrooms and ob- 
serve each other teaching them; they then hold debriefing sessions to reflect on and refine the les- 
sons. Once finalized, staff plan to compile the lessons in a “curriculum binder” that will serve as 
a resource tool to be designed and maintained by the school’s curriculum cadre. 

• Curriculum alignment across grade levels and increased teacher influ- 
ence on curriculum planning were the most frequently cited effects of the 
Accelerated Schools approach on curriculum and instruction. 

Given the varying interpretations and vague understanding that school staff reported re- 
garding practical applications of the powerful learning framework, no noteworthy trends were 
found in pedagogical techniques or curriculum choices across the eight sample schools. How- 
ever, there was consensus among staff at these schools that the Accelerated Schools approach 
had led to positive changes in curriculum and instruction. The area of change most consistently 
reported by the schools is the development of strategies to support curriculum alignment across 
classrooms and grade levels. 

For example, the curriculum cadre at one school led an effort to develop subject matter 
report cards that follow students from one grade to the next. Teachers are expected to assess stu- 
dents on particular skill areas, and the results are recorded on a report card that is designed spe- 
cifically to track how children are performing from one grade to the next. The report cards give 
teachers a record of the skill areas that were taught by teachers in previous grades, as well as each 
student’s performance in those areas. This tracking system is viewed as instrumental in enhanc- 
ing the school’s ability to align reading and communication arts skills across all grades. 

At another school, a cadre was formed to design school-wide approaches aimed at im- 
proving curriculum and instruction. This cadre began its work by proposing to (and gaining con- 
sensus from) the SAW that a consultant be hired to help staff develop a school-wide curriculum 
plan in order to provide continuity and strengthen connections among content areas across class- 
rooms and grade levels. The consultant helped teachers create units of study aligned with content 
and performance standards issued by the state Department of Education. A parent provided her 
perspective on how these activities had affected curriculum and instruction at the school: 



'^At the time of the site visit, this school was pilot-testing its powerful learning lab, and so curriculiun products 
generated from this activity were not yet available. In^lementation of powerful learning at this school receives 
strong principal and district-level support and is guided by an experienced Accelerated Schools coach. 
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The thing that impresses me most is the fact that there’s continuity now. Because 
there’s [always been] a lot of great teachers in this school . , . , but the thing I al- 
ways heard was that going from second [grade] to third or to fourth [the teachers] 
sort of did their own thing. There wasn’t time to find out what second [grade] was 
doing and third [grade] was doing. There wasn’t that continuity. There were pock- 
ets of good things here and there. I think in this process called Accelerated 
Schools there is continuity now, and to me that’s really important to giving kids a 
good education. 

Teachers at two other schools also reported that the Accelerated Schools approach fos- 
tered curriculum alignment and sharing of teaching techniques: 

I would say the main purpose of [the Accelerated School approach] was commu- 
nication between grade levels, learning what is going on in fourth grade and what 
do my kids need. ... I think it gave everybody a better [way] of sharing ideas be- 
tween grade levels and what is going on . . . just all that kind of coordinating. 

My first year [as a teacher here] I sank or swam on my own, but now teachers 
share things through the Accelerated Schools process. You know what’s going on 
in other classrooms and it helps you to know what should be going in your class- 
room. You feel like you’re not the only one having problems with something and 
you can leam from each other. . . . When I was teaching [at another nearby 
school], teachers were so protective and secretive about what material they were 
using and how they used it. Teachers who were known to be doing something in- 
novative were also the teachers in graduate school, trying to publish, and so they 
never wanted to share things. If you tried to use anyone else’s ideas, they’d kill 
you. Here, you are encouraged to share what you are doing. It’s always, “Come 
in” or “Sure you can watch me teach.” 

Increased teacher input into decisions affecting curriculum and instruction is a second 
area of change commonly linked to implementation of the Accelerated Schools approach. Such 
decisions include curriculum adoption, curriculum modification, and teacher training in how to 
implement new programs. For example, staff at one school described the impact of the Acceler- 
ated Schools approach on curriculum and instruction this way: 

One major change is in the way teachers view instruction. It is more of a decision- 
making process on the part of teachers. Prior to Accelerated Schools we were 
locked into a curriculum — you were given those curriculum guides and had to 
stick with that. No flexibility. When we became an Accelerated School, [we] 
talked about mastery reading and we were able to say, “Okay, this basal [reading 
curriculum] isn’t working and I want to use something else.” If that team of teach- 
ers felt [something else] would work for them, they were able to do it. That was 
one of the major changes. 

Teachers at another school reported that a similar process for input had developed for 
them as well: 
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Some things we do here are formulated, but there’s always room for you to shape 
it and make it work for our students. I think the [Accelerated Schools] structure 
helps a lot to make that happen. . . . The Chicago Math Program, we tried it out 
for a year and evaluated it. It was too heavily focused on manipulatives. What 
happened was that when students reached third grade and computation skills were 
required on paper, the kids didn’t have the foimdation. But we didn’t get rid of the 
program, we just enhanced it [by supplementing the program with a math text- 
book]. That’s what the [Accelerated Schools] process gives you — a way to im- 
prove on what you’ve started instead of saying, “This isn’t working; let’s scrap it 
for something new.” You’ll see that we have had the same [curricular] programs 
in place for a long time; we just keeping evaluating them and adding to them 
based on our experience and students’ needs. 

At another school, the ciuriculum cadre initiated a pilot math program, which was later 
adopted school-wide and then district-wide as test results from the school showed marked im- 
provement. A teacher described how using the Accelerated Schools process empowered them to 
make such decisions: 

You know the district likes everybody to do the same thing. ... [If it’s not work- 
ing for our students], we want to go out and find something else that is better for 
our kids. ... I think we have the confidence to do that and to stand up to [the dis- 
trict] I do think Accelerated Schools has helped foster that in us. 

Teachers in another school described the staff development model that they had designed 
using the Accelerated Schools approach. Based on a teacher survey indicating that district- and 
state-sponsored training workshops were not effective, the school spent a year holding discus- 
sions centered on staff development needs and interests. The result .was a school-designed pro- 
fessional development model that emphasized using teachers instead of consultants to provide 
staff training: “[We] said, don’t bring me a so-called expert. Bring me somebody that is actually 
doing this in the classroom.” The model stressed ongoing in-service instead of single-topic work- 
shops: “We used to have staff development like separated pieces of a jigsaw puzzle. Now, every- 
thing has to complement what we’re already doing. It has to come together and fit.” In 1998, this 
school received a national award for its staff development plan. 

V. ASP’s Technical Assistance Delivery Model 

The examination of the nature and extent of technical assistance that the sample schools re- 
ceived to facilitate their implementation of the Accelerated Schools approach focused on two 
things: (1) the schools’ access to an Accelerated Schools coach and (2) the nature and extent of the 
schools’ contact with ASP network resources (the ASP National Center and its regional Satellite 
Centers). Illustrative examples below indicate how the schools have used ASP technical assistance. 
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The key finding examined here is that no single standard approach to ASP technical assis- 
tance was in place at all the schools. Instead, three distinct variations of the ASP technical assis- 
tance delivery model were operating in these schools.'^ 

o Schools used a combination of local and network-based expertise to help 
guide and sustain their implementation of the Accelerated Schools ap- 
proach. 

As described in Chapter 2, the ASP technical assistance delivery model has evolved over 
time.''* Among the sample schools, three variations of the ASP technical assistance model were 
in effect. 

In variation 1, schools had a lot of contact with network resources during their first two 
years of implementation while also developing local technical assistance capacity to support im- 
plementation over time. Establishing this local capacity ultimately made school staff perceive 
that it was not necessary (and less useful) to continue strong ties with ASP network resources. 
Two schools in the sample appeared to have technical assistance of this type. 

One of these schools has received technical assistance fi'om a locally based Accelerated 
Schools coach since the initiative was launched in the fall of 1993. Its coach is a district adminis- 
trator who works part time in this capacity and was trained at the National Center to be an Accel- 
erated Schools coach. During the first three years of the initiative, the coach spent at least one 
full day a week at the school, facilitating school governance meetings, helping staff develop a 
consensus-based decision-making process, and guiding analysis of test score, survey, and other 
information used for taking-stock activities.'^ Comments fi'om staff indicate that they view this 
coach as integral to their maintenance of the Accelerated Schools approach: 

Most Accelerated Schools use a coach for a year or two. We have held on to our 
coach throughout. She’s like a sounding board for us. She helps present ideas to 
staff, is our direct contact to get support fiom the district, and plays devil’s advocate. 

She’s at all our cadre and SAW meetings. She comes when called. . . . The Acceler- 
ated Schools coach [has helped] determine what cadres will exist. She looked at 
specific needs of the school fiom its tests. . . . [The Accelerated Schools coach] has 
really helped to direct us to best practices and brings suggestions to the faculty. 



'^It is not certain which factors drove the variations that were found in the ASP technical assistance delivery 
model. Nor can this report say definitely what effects the variations had on the implementation or impacts of the Ac- 
celerated Schools approach. However, the developers of ASP currently advocate, as a prerequisite for implementa- 
tion of the initiative, that a district- or school-based staff person be provided release time to serve as Accelerated 
Schools coach. This shift fi'om the original technical assistance model is consistent with education research literature 
(Floden, Goertz, and O’Day, 1995), which indicates that local capacity-building is an essential element in growing 
and sustaining any school reform process. 

‘‘‘Early in the model’s evolution, the developers of the Accelerated Schools approach advocated that coaches 
(or trainers) be based outside the school, either fi'om the National Center or one of its regional Satellite Centers. 
The role of the coach at that time was to provide training and oversight that would enable schools to launch and 
carry out implementation of the Accelerated Schools approach. Typically, the coach would work with school staff 
over a two- or three-year period, after which time the school would be able to sustain implementation on its own. 

‘^The district administrator is now a division head at the district’s central office. As such, her time spent working 
as an Accelerated Schools coach has been reduced. She continues to work in this capacity at the school once a month 
and assists as requested by school staff. 
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In addition to the Accelerated Schools coach, staff at this school also received technical 
assistance from ASP trainers at a regional Satellite Center. When the school first began imple- 
menting the Accelerated Schools approach, the principal, the Accelerated Schools coach, and the 
curriculum specialist at the school attended bimonthly meetings hosted by the Satellite Center, fri 
turn, a trainer from the Satellite Center would visit the school almost monthly and provided two 
formal training sessions on the inquiry process. After two years, contact with the Satellite Center 
waned and is currently described by school staff as not very intensive: “Right now, contact with 
the Satellite Center is mostly through phone calls and we get the [ASP] newsletter.” A represen- 
tative from the Satellite Center visits the school three times a year: “[He] provides moral support 
and observes.” The school had not sent representatives to any of the ASP national conferences. 
The principal and the Accelerated Schools coach cited cost issues and the lack of usefulness as 
reasons why the school’s contact with the Satellite Center has changed: 

We’re not able to go the Satellite Center training activities as much because most 
of them require trips [out of town]. Also, when we do go, we had to do so much 
presenting ourselves it was a lot to prepare for and we weren’t getting as much 
back. . . . After a while, we stopped going to the monthly meetings because it 
seemed geared more toward schools at the beginning of the process. 



At a second school, the technical assistance experience evolved in a similar fashion. This 
school started its implementation of the Accelerated Schools approach with the assistance of a 
trainer provided through the ASP National Center. During the first two years of implementation, the 
trainer visited the school from two to five times aimually to conduct workshops and facilitate cadre 
meetings and the like, fri addition, during the summer, the school’s principal and a kindergarten 
teacher would attend turnkey training sessions at the National Center, fri year 2 of implementation, 
the kindergarten teacher was released half a day per week to serve as the Accelerated Schools 
coach, a role that she described as “99 percent of the time convincing people to give [the Acceler- 
ated Schools approach] a chance.” The principal described the coach as “someone who is not the 
principal, [but] who can force the issues and ask important questions.” However, both the principal 
and the coach reported that, after the second year of implementation, the school no longer received 
technical assistance support from the National Center. The school did send representatives to the 
ASP national conference when resources were available. According to the principal: 

We don’t have any connection with the Satellite Centers and we don’t get support 
from the National Center. We are probably providing more support to them than 
they are to us. Last summer, we had a powerful learning lab here at the school . . . 

[it] probably would not have happened without [our Accelerated Schools 
coach]. . . . We are lucky to have [her] as our own local expert. 

In variation 2 of the technical assistance model, schools developed local technical assis- 
tance capacity to support implementation, but they also maintained ongoing contact with ASP 
network resources. Such continued contact with the ASP network was considered helpful. Two 
schools in the sample appeared to have technical assistance of this type. 

Staff at one of these schools used a technical assistance approach that consisted of school- 
based Accelerated School coaches (referred to as “internal” coaches) as well as an “external” 
coach who is affiliated with a nearby regional Satellite Center. The work of each cadre at the 
school is facilitated by an internal coach — that is, teachers who help run the cadre meetings, co- 
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ordinate paperwork generated by the governance process (such as surveys, meeting materials, and 
agendas), post and update cadre meeting minutes on the teachers’ lounge bulletin board, and as- 
sist the external coach as needed. Ihtemal coaches typically serve as co-chairs for their cadres and 
(since 1996) have received college credit from a local imiversity where the Satellite Center is 
based. The external coach visits the school an average of once a week and serves to train staff 
(especially new teachers), advise the internal coaches, and facilitate school governance meetings. 
When asked to do so, the external coach also observed classes and did troubleshooting related to 
issues of school improvement planning. According to the external coach, this particular technical 
assistance model was school-initiated: “We use an internal and external coach[ing] model. The 
[strategy] was not endorsed by the [ASP] National Center.” Comments from the principal and a 
teacher at this school indicate that having both types of Accelerated Schools coaches is viewed as 
a valuable asset to the implementation process. 

The [external] coach has made a huge difference. You need somebody to crack the 
whip from the outside. You can’t do it from the inside alone. . . . Having an out- 
side coach was especially important for us. She keeps things going. . . . We’ve 
been doing [the Accelerated Schools approach] for a long time and it would be 
easy for the process to become stagnant. The coach keeps momentum going. . . . 

[The National Center] wanted us to have only the external coach, but [the princi- 
pal] thought we needed someone inside the school to help push the process. . . . 

We learned from the first year [of implementation] that we needed to build inter- 
nal capacity to support the process. 

At a second school in this category, three teachers serve part time as Accelerated Schools 
coaches. Staff reported that they also receive ongoing technical assistance support from a nearby 
affiliate office of the regional Satellite Center, which is housed at the state Department of Educa- 
tion. The school-based Accelerated Schools coaches attend weeklong summer retreats and meet- 
ings throughout the year hosted by the regional Satellite Center. 

In variation 3 of the technical assistance model, schools did develop ongoing local capacity 
to provide technical assistance on the Accelerated Schools approach. These schools reported fre- 
quent contact with ASP network resources during their first few years of implementation, although 
such contacts became more sporadic over time. Nonetheless, staff at these schools still perceive the 
network as a primary and valued source of technical assistance. Three schools in the sample 
adopted this variation of the technical assistance model, which is not surprising; this is the variation 
that most closely reflects the technical assistance delivery design initially advocated by ASP. 

At one school, the ASP network staff provided ongoing technical assistance during the 
first three years of implementation. Two Accelerated Schools coaches based at the state Depart- 
ment of Education and one trainer from the National Center visited the school on a regular basis. 
Collectively, these technical assistance providers conducted 10 in-service sessions during the 
school’s first year of implementation and continued to work two more years helping school staff 
to fme-frme processes for taking stock, conducting inquiry, and reaching consensus in decision- 
making. True to the original ASP model, support from these technical assistance providers to the 
school officially phased out in the spring of 1996. One of the Accelerated Schools coaches (who 
no longer works for the state Department of Education) maintains contact with the school. This 
former coach had discussions with the principal concerning the need to have a school-based 
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coach to ensure that implementation of the Accelerated Schools approach is maintained. School 
staff attended the annual ASP national conference when resources permitted.*^ 

Similarly, the second school in this category used an ASP regional Satellite Center as its 
primary source of technical assistance on implementing the Accelerated Schools approach. Dur- 
ing the first two years of the implementation process, the school received visits from a regional 
trainer every six to eight weeks. In the following year, contact became more sporadic, and school 
staff sensed a drop in momentum for the initiative. The principal laimched a “revitalization ef- 
fort” for the Accelerated Schools approach and requested additional assistance from the Satellite 
Center, which responded by sending a trainer almost weekly to address team-building skills and 
facilitate communications. Although the Satellite Center had responded quickly to the school’s 
requests for help, the principal reported that ongoing coaching from the start would have been 
beneficial in helping the school avoid pitfalls and stagnation in the implementation process. Staff 
in this school attended training and conferences hosted by the National Center to stay abreast of 
new developments. 

The evaluation sample consisted of schools that had adopted the Accelerated Schools model 
in the early to mid 1990s, an early phase in the model’s development and expansion. During that 
time, the model had clearly defined steps for building governance structures and school culture but 
was less clear in its methods to achieve powerful leamiiig. And although no one standard approach 
to ASP technical assistance was foimd, it was typical for staff across schools to report that the 
model’s available training was weakest in the area of powerful learning. Thus, the implementation 
study findings suggest that the schools in the study sample had met the maturity criteria for imple- 
mentation related to priority-setting, governance, and integration of Accelerated Schools principles 
and values. However, mature implementation of powerful learning was difficult to gauge because 
staff reports indicate that this concept was not a central focus of the training provided by ASP and 
had not been well defined. In part because of this, schools often focused on other, local concerns 
before devoting attention to reforming curriculxim and instruction. 



'®Three rqjresentatives from the school attended the 1999 ASP National Conference. 
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Chapter 5 

Impacts on Student Performance 



This chapter examines the impacts of the Accelerated Schools reform on a sample of 
elementary schools in which the reform had been launched in the early 1990s — early in the 
model’s development — and that had reached a mature level of implementation by the beginning 
of the evaluation. These early-generation programs were chosen because a reform with a process 
orientation, such as Accelerated Schools, must be implemented for several years before its effects 
can be evaluated. To measure the impacts of the reform on student performance, the standardized 
test scores in reading and math of third-grade cohorts in three consecutive baseline years (before 
the reform was launched) were compared with those of third-grade cohorts in five consecutive 
follow-up years (after the reform was laimched), as illustrated in Figure 1.1.' Each cohort in- 
cluded all third-grade students who were tested in a given school in a given year, and all the 
analyses controlled statistically for year-to-year changes in students’ backgroimd characteristics 
(as noted in Chapter 3, these differences were small). Once the impacts were estimated for each 
school, they were averaged across the eight study schools. 

The sections below summarize the impact findings in brief, outline how they were ob- 
tained, and explore them in greater detail. Owing to the complex nature of the analysis, the dis- 
cussion is sometimes technical, but the main findings can be understood on their own. 

I. Findings in Brief 

The following is a summary of the key impact findings, which hold for students’ test 
scores in both reading and math. 

• Impacts tracked implementation. In follow-up years 1 and 2, during which 
the schools focused on establishing supportive cultures and practices, there 
was no systematic change in average test scores. In follow-up year 3, when 
schools began to experiment with instructional changes intended to promote 
powerful learning, test scores declined somewhat. Finally, in follow-up years 
4 and 5, test scores gradually rose above the baseline level as instructional 
changes were implemented more fully. 

• By follow-up year 5, average reading and math scores were 0.19 standard 
deviation and 0.24 standard deviation above their respective baseline levels. 

These impacts are statistically significant and of modest size by conventional 



'Although five consecutive years of baseline data were available, the analyses reported here are based on only 
the three most recent baseline years. A conqianion to the present report (Bloom, 2001) lays out the methodology 
used for the present analysis in greater detail and explains why only three years of baseline data were used. The 
conqianion report also demonstrates that the inq>act estimates based on the three most recent years of baseline data 
are close to those based on all five years of baseline data. 
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standards.^ They are also comparable to the impacts of class size on student 
achievement found in the well-known Tennessee class-size experiment.^ Rela- 
tive to the populations against which students in the sample schools were 
scored, the present impacts reflect an overall increase in average reading per- 
formance of 6 percentile points, from the 37‘'’ percentile at baseline to the 43 
percentile in follow-up year 5; and an overall increase in average math per- 
formance of 7 percentile points, from the 46^'’ percentile at baseline to the 53'^** 
percentile in follow-up year 5. 

• The initially lowest-performing schools were the most likely to experience 
large impacts. The schools with the lowest test scores relative to their state or 
national norm at baseline were the most likely to experience targe relative 
gains in average test scores. 

• The impact of the Accelerated Schools reform appears to reflect mainly 
an increase in the scores of students who were in the middle of the test 
score distribution in their school. The reform did not appear to affect the 
scores of students who were at the lower end of the distribution. 

• Nevertheless, the reform improved the performance of otherwise low- 
performing students. At baseline, even students whose scores fell in the 
middle of their school’s distribution generally scored below the state or na- 
tional average. This was especially true of middle-performing students at the 
initially low-performing schools. 



II. Challenges for the Analysis 

For several reasons, it was not possible to construct a single readily interpretable measure 
of the impacts of the Accelerated Schools model on student performance for the eight schools in 
the sample. Because of this, the sections below present a series of impact estimates, each of 
which reveals a different facet of the reform’s effects. 

One major challenge for the impact analysis is that all the test scores used to measure stu- 
dent performance are standardized relative to a norming population, that is, are norm-referenced. 
Hence, they measure relative performance, not absolute performance expressed in terms of spe- 
cific skills acquired or achievement levels attained. Although the tests used by some of the 
schools in the sample were designed also to measure absolute performance — that is, are crite- 
rion-referenced — different tests did so in different ways. Thus, it was not possible to express 
the impact estimates for individual schools in terms of a common absolute benchmark or crite- 
rion such as a number of correct answers or a percentage of students who reached a specific level 



^As discussed later in this chapter, these findings are effect size estimates, which are used frequently to report the 
inqjacts of education (and other) programs. Their interpretation is based on benchmarks for assessing effect sizes 
proposed by Cohen (1977) and Lipsey (1990). 

^For recent summaries of effect size estimates from this study, see Nye, Hedges, and Konstantopoulos (1999) 
and Finn and Achilles (1999). 
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of performance. To make the magnitudes of the impact estimates interpretable, it was therefore 
necessary to develop ways of expressing impact estimates in relative terms. 

A second challenge for the impact analysis is that the sample schools administered differ- 
ent standardized tests, which in turn were “normed” with respect to — that is, scored against — 
different populations of students (however, each school administered the same test over the entire 
eight-year study period). Whereas half the schools used tests normed with respect to their state- 
wide population of third-graders, the rest used tests normed with respect to a national population 
of third-graders. Hence, the relative impacts of the Accelerated Schools reform on different 
schools’ average standardized test scores mean different things. Therefore, the impact estimates 
for each school had to be converted into a common metric to allow the estimates to be combined 
across schools. 

A third challenge for the impact analysis was that the Accelerated Schools reform, like 
other education initiatives, could in principle affect the distribution of student performance in any 
number of ways. For example, it might raise the performance of all students, across the full spec- 
trum of backgrounds and abilities, by the same amount (perhaps by motivating all students to try 
harder). If this were the case, the initiative would increase average performance without affecting 
its variability, that is, the disparities between students. In other words, the distribution of meas- 
ured student performance before implementation of the initiative would have exactly the same 
shape after implementation but would be shifted upward. A second possibility is that the initia- 
tive improves the performance of students whose backgrounds and abilities lie in the middle of 
the spectrum. In that case, the initiative would increase average performance as well as its vari- 
ability. A third possibility is that the initiative raises performance primarily for students with ttie 
weakest backgrounds or abilities (perhaps because they have the largest margin for improve- 
ment). In that case, the initiative would increase average performance and at the same time de- 
crease its variability by reducing the gap between lower- and higher-performing students. 

Owing to the range of possible impact patterns and the different policy implications of 
each, it was important to develop ways to distinguish among them. Doing so required estimating 
the impacts of the Accelerated Schools model on three measures of student performance: average 
test scores, the distribution of test scores, and the variability of test scores. Taken together, the 
results of these three analyses provide a more balanced picture of how the Accelerated Schools 
model affected the performance of students in this evaluation than could any single analysis. 

III. How Impacts Were Estimated 

The impacts of Accelerated Schools were estimated using a backward-looking inter- 
rupted time-series analysis. Backward-looking refers to the fact that schools in the sample were 
chosen retrospectively (after they had implemented the Accelerated Schools model). Interrupted 
time-series analysis refers to an evaluation design in which outcomes at multiple points before an 
intervention is launched (during its baseline period) are compared with those at one or more 
points after it is launched (during its follow-up period). In this study design, which has been used 
to evaluate many different kinds of programs,^ the outcomes observed at baseline are used to pro- 

^See, for exaiiqile, Cook and Canqjbell (1979). 
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ject what the outcomes would have been at follow-up in the absence of the intervention.^ The 
difference between actual and projected outcomes during the follow-up period provides an esti- 
mate of the impact of the intervention (what the intervention caused to happen). 

The present impact analysis focuses on third-grade achievement in reading and math as 
reflected in standardized test scores. These outcomes are highly relevant for judging the success 
of a primary school because reading and math are the two main areas of early academic devel- 
opment. The third grade also marks the midpoint of the elementary school experience, and third- 
grade performance is a good predictor of academic achievement in later grades. Furthermore, be- 
cause many elementary schools regularly test their third-grade students, it was possible to iden- 
tify a sample of schools that had records of third-grade scores on the same test for consecutive 
academic years. Lastly, by the third grade it was possible for some students in the sample to have 
been exposed to the Accelerated Schools model for up to three years, which should be enough 
time for an elementary school reform to produce impacts on performance. Table 5.1 lists the 
standardized tests used to measure third-grade performance, the norming populations against 
which the tests were standardized, and the calendar years comprising the three-year baseline pe- 
riod for each school in the sample. 

As explained in Chapter 3, the schools in the present evaluation were chosen partly for 
having experienced no or few major changes, such as redistricting or implementation of other 
reforms, during the study period that could affect the pattern of student test scores over time. 
Nevertheless, one or two schools experienced shifts over time in the background characteristics 
of their third-grade students. To account for these shifts, changes in the background characteris- 
tics for which data were available were controlled for statistically by including them as inde- 
pendent variables in a regression model that was used to estimate impacts fi-om the interrupted 
time-series data.^ Thus, all the impact estimates presented in this report are regression-adjusted 
differences between follow-up outcomes and baseline outcomes.^ The full list of background 
characteristics used for these analyses were gender, race/ethnicity, whether the student was over- 
age for the third grade, fi-ee/reduced-price lunch status, special education status, and whether 
English was the student’s first language. Because some schools in the sample did not have data 
for all these characteristics, different background characteristics were used in the regression 
models for different schools. Table 5.2 indicates which characteristics were used for each school. 

IV. Impacts on Average Test Scores 

A first step in assessing the impacts of the reform on the mature, early-generation Accel- 
erated Schools in the present sample is to estimate its impacts on average, or mean, reading and 
math test scores. 



*This hypothetical outcome is usually referred to as a counterfactual. 

*See Bloom, 2001. 

^Because most schools in the sanple experienced little or no shift over time in the background characteristics of 
their third-grade students, the regression adjustments had little effect on the intact estimates (see Bloom, 2001). 
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Accelerated Schools Evaluation 
Table 5.1 

Standardized Test Used by Each School in the Sample 



School 


Name of Test 


Calendar Baseline Years^ 


Nonning Population 


Sheppard, CA 


Con^rehensive Test of Basic Skills 4 


1988-1990 


National 


PS 108, NY 


Degrees of Reading Power (reading) 


1991-1993 


State 




California Achievement Test (math) 5 




National 


Memminger, SC 


Basic Skills Assessment Program 


1989-1991 


State 


Blanche Pope, HI 


Stanford Achievement Test 8 and 9 


1991-1993 


National 


Lucy W. James, MO 


State Mastery and Achievement Test 


1990-1992 


State 


Shepard, MO 


Stanford Achievement Test 


1990-1992 


National 


McCleery, IL 


CoiT^rehensive Test of Basic Skills 


1991-1993 


National 


Mary Wright, SC 


Basic Skills Assessment Program 


1991-1993 


State 



SOURCES: 

Basic Skills Assessment Program, A UsePs Guide to BSAP Score Reports , South Carolina State Board of Education, 
1998, pp. 2-4. 

California Achievement Tests , Fifth Edition, McGraw-Hill, 1996, p. 16. 

Comprehensive Tests of Basic Skills Technical Report Edition, Macmillan/McGraw-Hill, 1991, p. 87. 

The DRP: An Effectiveness Measure in Readings College Entrance Exam Board, 1987, pp. 137-139. 

State Mastery and Achievement Tests, Technical Manual , South Carolina State Department of Elementary 
Education, 1998, p. 13. 

Stanford Achievement Test Series Spring Norms Book ^ Ninth Edition, Harcourt Brace, 1997, p. 1 1 . 

NOTES: In order to preserve the confidentiality of schools* test score results, the order of sites in this table differs firom 
that of subsequent tables which identify schools by letters (A-H). 

“Years listed represent the spring semesters of the baseline school years. 
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Table 5.2 

Demographic Characteristics Controlled for at Each School in the Sample 



School 


Overage for 
Grade" 


Gender 


Ethnicity 


Free or Reduced- 
Price Lunch Status 


Special Education 
Status 


Repeater 

Status^ 


Language 


A 


✓ 


✓ 




✓ 


✓ 






B 


✓ 


✓ 






✓ 


✓ 




C 


/ 


✓ 


✓ 


✓ 




✓ 




D 




✓ 


✓ 


✓ 




/ 




E 


/ 


/ 


/ 




✓ 


/ 




F 


/ 


/ 


✓ 






/ 




G 


/ 


/ 


/ 


✓ 




/ 




H 




/ 


/ 











SOURCE: Student demographic data provided by each school. 

NOTES: Estimates were adjusted using ordinary least squares to control for demographic characteristics of san^le 
members. 

®Age was calculated as of April 15 of the spring semester. 

^^Repeater status was determined by whether the same individual student identifier appeared in two or more 
consecutive years. 
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A. Obtaining the Estimates 



These estimates were calculated separately for each follow-up year at each school. The 
estimates for each follow-up year were then averaged across schools.^ For example, the impact 
on the average math test score at a particular school in follow-up year 1 was estimated as the dif- 
ference between its average math score in that year and that during the baseline period.® The im- 
pact on each school’s average test scores in each subject area in each year was likewise obtained 
by comparing the average with its baseline counterpart. 

Estimates of both the magnitude and the statistical significance of the Accelerated School 
reform’s impacts on average test scores were obtained in this manner. However, owing to the 
small size of the student samples at each school and the limited ability of interrupted time-series 
analysis to rule out alternative explanations for test score changes at a single school, the findings 
for individual schools are not as reliable or valid as those for the full sample of schools. As a re- 
sult, the focus here is on full-sample impacts rather than impacts on individual schools. 

To average the impact estimates across schools, which administered different tests, the 
scores were first converted to a common metric using standard procedures for combining impact 
estimates across studies.'® For a given school, the first step was to express the impact estimate 
for each follow-up year as a multiple of the standard deviation of that school’s individual test 
scores during the baseline period." In this way, all the impact estimates were transformed into 
standard deviation units. Thus, for example, an impact estimate of 0.25 indicates that the average 
test score for a particular follow-up year lay 0.25 standard deviation above the corresponding av- 
erage for the baseline period. Because these impacts are expressed in standard deviation units, 
they can be combined across schools. This type of impact estimate, which is used widely in 
evaluation research, is referred to as an effect size. 

Although judgments about whether a specific effect size is large or small are ultimately 
arbitrary, some guidelines do exist. For example, many researchers use the rule of thumb that ef- 
fect sizes of around 0.20 are small, around 0.50 moderate, and more than 0.80 large.'^ Empirical 
support for this categorization was provided by an analysis of the distribution of 102 average 
effect sizes from 186 syntheses of treatment effectiveness studies, most of them drawn from edu- 
cation research. The bottom third of this distribution (small effect sizes) ranged from 0.00 to 
0.32; the middle third (moderate effect sizes) ranged from 0.33 to 0.55; and the top third (large 
effect sizes) ranged from 0.56 to 1.26. 



^Schools were weighted equally in calculating these means because the analysis focused on in^acts for the aver- 
age school in the san^le. For a rationale, see Bloom (2001). 

^The analysis was originally to entail fitting a regression-adjusted trend line through the five years of baseline 
test scores for each school and using this trend line to project for each school in each follow-up year what the mean 
score would have been without the Accelerated Schools initiative. However, in almost all cases, the slopes of these 
baseline trends were not statistically significantly different from zero (see Bloom, 2001). Therefore, all projections 
for the follow-up period were based on regression-adjusted baseline mean scores. 

’°See, for exan^le. Hedges and Olkin (1985). 

”The unadjusted standard deviation of all test scores for the three-year baseline period at each school was used 
for this purpose (separately for reading and math). 

’^This rule of thumb was proposed by Cohen (1977). 

’^Lipsey, 1990. 
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Another benchmark against which to compare the effect size estimates in this evaluation 
is the findings from the Tennessee class-size experiment mentioned earlier in this chapter. This 
study is a relevant point of reference for three reasons: (1) Like the present evaluation, it exam- 
ined a major effort to improve student performance at the elementary school level; (2) it esti- 
mated program impacts using a randomized experiment, the most valid way to find out what dif- 
ference an intervention makes; and (3) it used a definition of effect size comparable to that used 
here.*'* In the Tennessee experiment, the intervention was a reduction in class size from the stan- 
dard range of 22 to 26 students to a range of 13 to 17 students. This reduction had effects on av- 
erage reading and math test scores of between 0.15 and 0.25 standard deviation.*^ 

B. Findings 

Once the impact estimates for the individual schools were transformed into the same 
units, it was possible to compute the overall impact across schools in each follow-up year and to 
determine the statistical significance of each average. Figure 5.1 illustrates the results of this 
“pooled” impact analysis for reading test scores. Each point in the figure represents the average 
reading score for a specific year relative to the overall average score in the baseline period. The 
distance between each point and the nearest point on the horizontal axis represents (in standard 
deviation units) how much it differs from the baseline average. 

First consider the pattern of reading scores during the baseline period. The scores varied 
somewhat from year to year around the baseline average, but they were generally quite stable.*^ 
This stability reflects the fairly large sample of students whose scores went into the average cal- 
culated for the full sample of schools and the fact that year-to-year fluctuations for individual 
schools tend to average out when the results for multiple schools are combined. 

Next consider the pattern of scores during the follow-up period. In follow-up years 1 and 
2, there was virtually no change relative to the average baseline level. In follow-up year 3, how- 
ever, there was a statistically significant decline in the average reading score of -0.15 standard 
deviation relative to the average baseline level.*’ In follow-up years 4 and 5, the average score 
gradually reboimded, lying 0.19 standard deviation above the baseline average in follow-up year 
5 (this difference was statistically significant). 

Now consider the corresponding findings for math test scores, which are summarized in 
Figure 5.2. Once again, note that during the baseline period the average scores for each year 



'^Comparing effect size estimates across studies that measure program in^acts in different ways can lead to se- 
rious problems (see Olejnik and Algina, 2000). As explained in Chapter 6, it is therefore inappropriate to compare 
effect size estimates from the present study with those from analyses of program impacts based on student growth 
curve models (for exanq)le, Ross, Sanders and Wright, 2000). 

*^See, for exanq)le, Finn and Achilles (1999); and Nye, Hedges, and Konstantopoulos (1999). 

‘®A corresponding analysis including all five baseline years indicates that the average reading scores varied 
slightly from year to year but exhibited no systematic trend over time (Bloom, 2001). 

*’The statistical significance of an impact estimate indicates whether it is likely to be “real” as opposed to a re- 
sult of random sanq)ling or estimation error. Statistical significance is not necessarily an indicator of the magmtude 
or importance of an effect. For instance, even a very small effect can be statistically sigmficant if found in a suffi- 
ciently large sanq)le. 
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fluctuated around the baseline average. This stability was followed by modest fluctuations during 
follow-up years 1 and 2. In follow-up year 3, the average math score, like the corresponding read- 
ing score, fell below the baseline average. In follow-up years 4 and 5, however, the average score 
rebounded: By follow-up year 5, it was 0.24 standard deviation above the baseline average (this 
difference was statistically significant). 

C. Interpretation of the Findings 

The foregoing findings are consistent with the implementation results presented in Chap- 
ter 4. During the first two years of implementation, the focus of attention was on changing school 
governance, decision-making, and culture. It was only later that attention was turned to changing 
instructional practices. However, there was confixsion about what the instructional component of 
the Accelerated Schools model (powerfixl learning) was supposed to be, and it took time to im- 
plement this feature of the reform. 

In light of this, perhaps the most plausible interpretation of the combined pattern of im- 
pact and implementation findings is the following. In follow-up years 1 and 2, few or no changes 
were made to instructional practices, and thus few or no systematic changes in student perform- 
ance were observed. By follow-up year 3, as schools began to change their instructional practices 
but had not yet worked out the details, instruction might actually have been less effective for stu- 
dents than it had been previously, and — perhaps as a result — student performance declined 
temporarily. In follow-up years 4 and 5, however, as curricular and teaching changes began to 
take hold and faculty learned how to use the new materials and approaches that had been 
adopted, student performance began to improve. 

By follow-up year 5, the average test scores in reading and math exceeded those during 
the baseline period by 0.19 and 0.24 standard deviation, respectively. These impact estimates fall 
in the small to modest range by conventional standards and are of a size comparable to the 
achievement gains found in the Tennessee class-size experiment. 

D. Findings by School 

Although the exact estimates of impacts for each follow-up year varied considerably fi'om 
school to school, the basic pattern of impacts for the whole sample over time is also reflected in 
most of the school-level impacts. In other words, the pattern across schools is not driven by im- 
pacts in just one or two schools. 

To see this point, consider the impact estimates by school and follow-up year presented in 
Tables 5.3 and 5.4. The bottom row in each table shows the average impact estimate for each 
year, with each school weighted equally. Most schools experienced a decline in test scores in fol- 
low-up year 3, with a gradual increase thereafter. Not all the schools attained a positive and sta- 
tistically significant impact in follow-up year 5, but the overall average impact in this year was 
positive and statistically significant. 

E. Findings by School Relative to State or National Norms 

To place the impact findings in a broader context, the estimates of the average baseline 
and follow-up scores for each school are expressed in terms of each school’s state or national 
percentile equivalent in Table 5.5. (School C had no test score data for follow-up year 5 and is 
therefore not included in the table.) The main findings appear below. 
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Accelerated Schools Evaluation 
Table 5.3 



Impacts on Mean Third-Grade Reading Scores 



School 




Standardized Impact (Effect Size) per Follow-Up Year 




Year 1 




Year 2 


Years 


Year 4 


Years 


A 


-0.32 




-0.17 


-0.43 ** 


-0.26 


0.05 


B 


0.00 




-0.09 


-0.28 *’*' 


-0.16 


-0.06 


C 


-0.11 




-0.31 


-0.03 


-0.22 


N/A 


D 


-0.05 




-0.39 *** 


-0.23 


-0.17 


-0.27 * 


E 


0.16 




0.39 ** 


-0.28 * 


-0.24 


0.15 


F 


0.36 




0.28 * 


0.54 *** 


1.05 *** 


0.98 *** 


G 


-0.35 




0.06 


0.02 


0.30 * 


0.43 *** 


H 


-0.05 




0.39 ** 


-0.49 *** 


-0.16 


0.06 


Average 


-0.04 




0.02 


-0.15 ** 


0.02 


0.19 *** 



Accelerated Schools Evaluation 
Table 5.4 

Impacts on Mean Third-Grade Math Scores 


School 


Standardized Impact (Effect Size) per Follow-Up Year 


Year 1 


Year 2 


Years 


Year 4 


Years 


A 


-0.19 


0.34 


-0.56 


-0.08 


0.50 


B 


-0.01 


0.03 


-0.06 


-0.24 


0.00 


C 


0.23 


-0.37 


-0.39 


0.05 


N/A 


D 


-0.22 


-0.31 


-0.12 


0.05 


-0.04 


E 


0.35 


0.51 * 


0.32 


0.51 * 


0.24 


F 


-0.09 


0.19 


0.07 


0.21 


0.32 


G 


-0.26 


0.41 


0.31 


0.33 


0.69 *** 


H 


-0.16 


-0.04 


-0.27 


-0.28 


-0.06 


Average 


-0.04 


0.10 


-0.09 


0.07 


0.24 ** 


(continued) 
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Tables 5.3 and 5.4 (continued) 



SOURCE: MDRC calculations using student test scores and demographic data provided by each school. 

NOTES: The sample includes all third-grade students tested during the study period at each school. 

Estimates were adjusted using ordinary least squares to control for demographic characteristics of 
sanq)le members. 

A two-tailed test was applied to differences between the baseline and follow-up test scores. Statistical 
significance levels are indicated as ***= 1 percent, **= 5 percent, *= 10 percent. 

Findings in these tables represent equally weighted averages in which all schools received the same 
weight regardless of sample sizes. 
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Accelerated Schools Evaluation 
Table 5.5 



The Percentile Equivalent of Each School’s Mean Baseline and Follow-Up Scores 
Relative to Its State or National Norming Population 



School® 


Reading Percentile 




Math Percentile 


Baseline Period Follow-Up Year 5 


Baseline Period 


Follow-Up Year 5 


A 


21 


23 


36 


50 


B 


54 


51 


53 


45 


D 


56 


48 


63 


61 


E 


43 


48 


45 


53 


F 


24 


54 


43 


57 


G 


26 


39 


31 


56 


H 


43 


45 


49 


47 


Average*^ 


37 


43 


46 


53 



SOURCE: MDRC calculations using student test scores and demographic data provided by each school. 

NOTES: Estimates were adjusted using ordinary least squares to control for demographic characteristics of 
san^le members. 

“This table includes findings only for the seven schools with test score data for follow-up year 5. 

*The percentile rank for the average school was obtained by (1) converting the percentile score for the 
average student at each school to a Normal Curve Equivalent (NCE), (2) confuting the mean NCE for all 
schools, and (3) converting the mean NCE back to percentiles. This process thus uses the cardinal properties of 
NCEs for confuting means and the ordinal properties of percentiles for reporting fmdings. 





• During the baseline period, third-graders whose scores were average in 
their own school typically scored below average — and, in some schools, 
far below average — statewide or nationally. The average baseline scores 
for five schools were at least 5 percentile points below the average of their 
norming population in both reading and math. 

• By follow-up year 5, the average score for each school in each subject was 
noticeably closer to its corresponding state or national average. By the end 
of the follow-up period, the number of schools scoring five or more percentile 
points below their norming group’s average was only three in reading and one 
in math. Furthermore, five schools had average math scores at or above their 
norming group’s average. 

• On average, the schools increased their students’ relative performance by 
6 percentile points in reading and 7 percentile points in math during the 
five-year follow-up period. The average reading score increased fi'om the 
37*'’ percentile during the baseline period to the 43*** percentile in follow-up 
year 5. The corresponding increase in the average math score was fi-om the 
46*'’ percentile to the 53**' percentile. 

• The lowest-performing schools at baseline experienced the largest im- 
provements in test scores. Schools F and G, whose average baseline reading 
scores were at the 24*'’ percentile and 26*'’ percentile relative to their respective 
norming populations, experienced the largest percentile improvements in read- 
ing performance. Likewise, the largest relative improvements in math per- 
formance were observed at schools A, E, F, and G, whose average baseline 
math scores were at the 36*'’ percentile, 45*'’ percentile, 43*** percentile, and 31** 
percentile relative to their respective norming populations. 



V. Impacts on the Distribution of Test Scores 

Estimates in the previous section represent the impacts of the Accelerated Schools initia- 
tive on average student performance. However, as discussed earlier, it is possible that differ^t 
types of students responded differently to the initiative. If this were the case, then focusing exclu- 
sively on how average performance was affected would mask important variation that could shed 
light on the reform’s effects. This section therefore estimates the impact of the Accelerated 
Schools model on the distribution of student performance and considers where in the distribution 
of student performance the observed impacts were most likely experienced. 

A. The Baseline Distribution 

As a first step in this process. Figures 5.3 and 5.4 show the reform’s estimated impacts on 
the distribution of test scores. The first vertical bar in Figure 5.3 represents the distribution of 
individual reading scores for all the third-grade students in the baseline cohorts. For each school, 
the scores were divided into three categories: (1) those in the upper baseline quartile (the upper 
25 percent of all baseline scores), (2) those in the middle two baseline quartiles (the middle 50 
percent of all baseline scores), and (3) those in the lower baseline quartile (the lower 25 percent 
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NOTES: The sample includes all third-grade students tested during the study period at the eight Accelerated Schools selected for evaluation. 

Estimates were adjusted using ordinary least squares, controlling for demographic characteristics of sanq)le members. 

A two-tailed test was applied to differences between the baseline and follow-up test scores. Statistical significance levels are indicated as ***= 1 percent, 
**= 5 percent, *= 10 percent. 

Rounding may cause slight discrepancies in the calculations of siuns and differences. 
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of all baseline scores). Hence, by definition, 25 percent of the baseline scores were in the upper 
baseline category, 50 percent were in the middle baseline category, and 25 percent were in the 
lower baseline category. 

When fi’aming the analysis in this way, it is important to note that students who scored in 
the upper baseline category in their school were not necessarily above average statewide or na- 
tionally, that is, relative to their norming population.** Rather, they scored above the average of 
other third-graders in their school during the baseline period. Because the average baseline score 
for most schools in the present sample was below the state or national average, many students in 
the upper baseline category probably were not above average statewide or nationally. 

B. Chan2es in the Overall Distribution 

Defining the upper, middle, and lower baseline categories for each school with respect to 
its distribution of test scores diuing the baseline period provides a consistent fi’amework for 
gauging subsequent changes in this distribution. First, for each follow-up year the percentage of 
students in each school whose test scores were in each baseline category was computed. Then the 
percentage of students in each category was averaged across all schools.*® 

Figme 5.3 illustrates the results of this analysis for reading test scores. As can be seen, 
there was not much change in the distribution of reading scores during the first two follow-up 
years. The small changes that did occur comprised increases of 2 to 3 percentage points in both 
the upper and lower categories, consistent with the finding that average reading scores fluctuated 
little during the first two follow-up years. 

In follow-up year 3, however, there was a statistically significant decrease in the percent- 
age of students scoring in the upper baseline category and a statistically significant increase in the 
percentage scoring in the lower baseline category. This finding is consistent with the decline in 
average test scores in follow-up year 3. 

The situation began to improve thereafter. Thus, by follow-up year 5, 37 percent of the 
students scored in the upper baseline category, as opposed to 25 percent during the baseline pe- 
riod. This 12 percentage point increase was statistically significant. Given that there was no 
change over the same period in the percentage of students scoring in the lower baseline category, 
the increase in the percentage in the upper baseline category appears to have been produced en- 
tirely by a corresponding decrease in the percentage of students scoring in the middle baseline 
category (fi’om 50 percent to 38 percent). 

Figme 5.4 presents a similar picture for math test scores. First, there was a modest down- 
ward shift in the distribution of test scores in follow-up year 3, with slightly fewer scores in the 
middle baseline category and slightly more in the lower baseline category. Second, there was a 



’®The baseline categories were not defined in terms of where students scored relative to their state or national 
coimterparts because the differences in the schools’ norming populations would have made it extremely difficult to 
summarize findings across schools. 

”For example, in follow-up year 5 the regression-adjusted percentages of students whose math scores were in 
the upper baseline category in the seven schools for which follow-up year 5 data are available are 43 percent, 24 
percent, 23 percent, 42 percent, 46 percent, 53 percent, and 23 percent. The mean, 36 percent, is the corresponding 
percentage of scores in this category for the average school in the sample. 
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noticeable upward shift in follow-up year 5, which appears to have been due almost entirely to an 
increase in the percentage of scores in the upper baseline category (from 25 percent to 36 per- 
cent) and a corresponding decrease in the percentage of scores in the middle baseline category 
(from 50 percent to 40 percent). Also, in follow-up year 5, there was very little change (from 25 
percent to 23 percent) in the percentage of scores in the lower baseline category. 

In sum, it appears that virtually all of the improvement in average test scores — both in 
reading and in math — was experienced by students who would have scored in the middle base- 
line category without the Accelerated Schools reform.^® There was very little sign of improve- 
ment at the low end of each school’s performance distribution.^* Nevertheless, most of the im- 
pacts of the reform were experienced by otherwise low-performing students, because in the 
schools that experienced the largest impacts, students who scored in the middle of their school’s 
performance distribution were typically well below the statewide or national average. 

C. Changes in the Distribution bv School 

Tables 5.6 and 5.7 expand on the preceding findings by listing the percentage of students 
by school who scored in the upper baseline category during each follow-up year.^^ Recall that, by 
definition, 25 percent of all baseline scores fell in this category. Thus, an increase relative to 25 
percent in a given follow-up year represents an improvement and a decrease relative to 25 per- 
cent represents a decline. 

The most striking and consistent findings are for follow-up year 5: Four of the seven 
schools witnessed noticeable improvements in reading (with 33 percent, 38 percent, 51 percent, 
and 63 percent of their students scoring in the upper baseline category), and three of these in- 
creases were statistically significant.^^ The remaining schools exhibited virtually no change (with 
23 percent, 26 percent, and 26 percent in the upper baseline category). Although the specific 
schools that experienced improvements were different, the findings are essentially the same for 
math. Four of the seven schools exhibited marked improvements (with 42 percent, 43 percent, 46 
percent, and 53 percent of their students scoring in the upper baseline category), and three of 
these increases were statistically significant.^^ The other three schools experienced virtually no 



is not possible on the basis of existing data to detennine with certainty from where in the distribution the 
additional students who scored in the upper baseline category at the end of the follow-up period came. However, to 
conclude that they came from the lower baseline category would require assuming that an equal number of students 
moved down from the middle to the lower baseline category, which seems iiiqjlausible. 

^'it is possible that the reform iiiqjroved scores in the lower baseline category somewhat but not enough to move 
them into &e middle baseline category. In addition, it is possible that the reform increased scores in the upper base- 
line category, which could not be revealed by the present analysis. However, given the modest mqjrovement in aver- 
age scores and the marked increase in the percentage of students scoring in the upper baseline category, it seems 
most likely that almost all the test score increases produced by the Accelerated Schools model were experienced by 
students who would have scored in the middle baseline category without the model. 

“Appendix Tables A.1-A.4 provide corresponding findings for the middle and lower baseline categories. 

“it is possible that four unrelated events produced the inqjrovements observed in these schools. However, four 
different, unspecified explanations are less plausible than the single hypothesis that the Accelerated Schools reform 
led to inqjrovements that it was designed to produce. 

“Given the small sanq>le size for each school, only very large percentage changes at the school level are statisti- 
cally significant. 
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Table 5.6 



Percentage of Reading Scores in the 
Upper Baseline Category for Each Follow-Up Year 



School 


Year 1 


Year 2 


Years 


Year 4 


Years 


A 


18 


21 


11 * 


13 


26 


B 


30 


23 


15 ** 


25 


26 


C 


21 


16 


18 


18 


N/A 


D 


26 


13 ** 


17 


27 


23 


E 


40 ** 


47 *** 


14 * 


22 


33 


F 


42 *** 


37 ** 


53 *** 


68 *** 


63 *** 


G 


12 ** 


25 


27 


30 


51 *** 


H 


30 


39 ** 


17 


22 


38 ** 


Average 


27 


28 


21 * 


28 


37 *** 



Accelerated Schools Evaluation 
Table 5.7 

Percentage of Math Scores in the 
Upper Baseline Category for Each Follow-Up Year 


School 


Year 1 


Year 2 


Year 3 


Year 4 


Years 


A 


20 


44 * 


5 


21 


43 


B 


25 


31 


27 


22 


24 


C 


36 


19 


16 


35 


N/A 


D 


11 


15 


20 


26 


23 


E 


45 * 


36 


38 


47 ** 


42 * 


F 


22 


32 


28 


32 


46 ** 


G 


19 


42 * 


40 


37 


53 *** 


H 


16 


25 


21 


15 


23 


Average 


24 


31 


24 


29 


36 *** 



(continued) 
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Tables 5.6 and 5.7 (continued) 



SOURCE: MDRC calculations using student test scores and demographic data provided by each school. 

NOTES: The san^le includes all third-grade students tested during the study period at each school. 

Estimates were adjusted using ordinary least squares to control for demographic characteristics of sanple 
members. 

A two-tailed test was applied to differences between the baseline and follow-up test scores. Statistical 
significance levels are indicated as ***= 1 percent, **= 5 percent, *= 10 percent. 

Findings in these tables represent equally weighted averages in which all schools received the same weight 
regardless of sanq)le sizes. 



ERIC 



-TO- 



SS 



change (with 23 percent, 23 percent, and 24 percent of their students scoring in the upper base- 
line category). 

Based on the distribution analysis, it appears that the observed gains reflect positive 
changes in a majority of the seven schools in both reading and math. 

VI. Impacts on the Variability of Test Scores 

To flesh out the interpretation of the impacts of the Accelerated Schools reform further, 
the variability of test scores was examined next. Variability was expressed as the standard devia- 
tion of each school’s individual test scores. To capture the change in test score variability from 
the baseline period to a given follow-up year for a specific school, the standard deviation of the 
scores for each follow-up year was divided by its baseline counterpart.^^ A result of greater than 
1 indicates an increase in variability, and a result of less than 1 indicates a decrease in variability. 

Recall that the increase in average reading and math scores observed in follow-up year 5 
was produced by an increase in the percentage of scores in the upper baseline category, a corre- 
sponding decrease in the middle category, and virtually no change in the lower category. Thus, 
the distribution of reading and math scores in this year was wider than it was during the baseline 
period, meaning that the variability of scores had increased. More specifically, the estimated 
standard deviation of the test scores in follow-up year 5 was 13 percent larger than its baseline 
counterpart for reading and 1 1 percent larger for math.^® Thus, by apparently affecting mainly 
students who were in the middle of the baseline distribution, the Accelerated Schools reform, as 
it was implemented in the schools in this evaluation, somewhat increased the disparities in stu- 
dents’ test performance. 

VII. Implications of Student Mobility for the Findings 

Student mobility, which is especially high in urban areas, can reduce the amount of time 
that some students are exposed to an education initiative and thus dilute its impacts. For exam- 
ple, toward the last two years of the follow-up period, by which time the Accelerated Schools 
model had been almost fully implemented, between 7 percent and 1 8 percent of the students who 
were enrolled in one of the study schools at the beginning of an academic year were no longer 
enrolled there at the end of the year (this percentage is the fall-to-spring move-out rate)}^ To ex- 
trapolate the implications of this pattern over multiple years requires knowing the year-to-year 



described in Bloom (2001), this was accomplished by (1) estimating, for each follow-up year and for each 
school, the variance of the residuals of the regression used to measure inqjacts on average student performance; (2) 
estimating the corresponding variance of the baseline residuals; (3) using the ratio of these two estunated variances 
as an F statistic to test the statistical significance of their difference; and (4) using the square root of this ratio to es- 
timate the corresponding ratio of standard deviations. 

Appendix Tables A.5 and A.6 present the corresponding findings for each follow-up year both by school and 
for the full sample of schools. 

^’The fall-to-spring move-out rates in the six schools for which appropriate data were available were approxi- 
mately 7 percent, 9 percent, 15 percent, 16 percent, 16 percent, and 18 percent. One school, which was located near 
a military base, reported that, though it did not keep precise records on mobility, its move-out rate was quite high. It 
was not possible to determine the mobility rate in the remaining school. 
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move-out rate, which in turn requires knowing how many students left the study schools over the 
summer. Unfortunately, this information is rarely tabulated by schools and was not available for 
the present study. However, the estimated year-to-year move-out rate of 20 percent in the Chi- 
cago public school system provides a rough approximation of typical move-out rates in a major 
urban area.^^ 

If an Accelerated School’s fall-to-spring move-out rate is 15 percent (the median rate in 
the study sample) and its year-to-year move-out rate is 20 percent (the average rate in Chicago 
public schools), then, by the end of follow-up year 3, 85 percent of its third graders would have 
been exposed to the reform for at least one full academic year; 68 percent would have been ex- 
posed for at least two full years; and 54 percent would have been exposed for at least three full 
years. Thus, most but not all of the students in the study probably received a substantial dose of 
the reform during the later follow-up years, when it was largely in place. 

To account properly for the fact that not all students did or will receive a full dose of the 
Accelerated Schools model (or any other multiyear education initiative), it is important to distin- 
guish between two different impact questions. 

1. What were the impacts of Accelerated Schools on the typical third-grader 
when the model was implemented under real-world conditions (which include, 
among other things, student mobility)? 

2. What impacts does the initiative have on third-graders who remain in the same 
school long enough to receive a substantial “dose”? 

The first question focuses on the effectiveness of Accelerated Schools in the real world, at 
least as the real world now exists. This question, which the present evaluation addresses directly, 
is relevant for deciding whether to promote broad application of the model. 

The second question focuses on the potential effectiveness of Accelerated Schools if it 
could be operated in a world where students did not change schools or if all schools adopted the 
model such that all students would be exposed to it regardless of where they went to school. This 
question is relevant for modifying the model to increase its effectiveness. Based on the data 
available for the present study, there is no way to address this question directly. However, be- 
cause most students in the present sample received more than one full year of exposure to the re- 
form once it was finally implemented (during follow-up years 4 and 5), the present findings may 
provide an approximate answer to the second question, although the impacts found are probably 
smaller than they would be under conditions of maximum dosage. 

Owing to the difference between the two impact questions above, and the different kinds 
of data required to address them, one should be very careful when comparing the present findings 
with those of studies that measure the impacts of comprehensive school initiatives only on stu- 
dents who receive extensive program exposure. The latter kind of investigation is exemplified by 
evaluations that focus on the “growth curves” of test scores across several grades for a group of 



^*Bryk, Thum, Easton, and Luppescu, 1998. 
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students who attend the same school for all the grades imder study and therefore receive substan- 
tial exposure to the program.^^ 

There are two important differences between findings from growth-curve studies and 
those from studies like the present one.^° First, growth-curve analyses estimate the impact of 
lengthy program exposure, whereas the present study estimates the impact of a mix of exposure 
lengths. If greater exposure produces larger impacts, then growth-curve analyses will yield larger 
impact estimates for the same program. A second source of noncomparability arises from differ- 
ences in the characteristics of the students represented in the two types of studies. The present 
study represents all students in a school, regardless of whether they changed schools during the 
study period, whereas growth-curve analyses include only students with very low mobility. If the 
impacts of a program are greater for students with stable home (and thus school) environments, 
then impact estimates based on growth-curve analyses will be systematically larger than those 
based on studies such as the present one.^' 



^’See, for exanqjle, Ross, Sanders, and Wright (2000). 

third, unrelated source of nonconq)arability stems from differences in the defrnitions of inqiacts used in the 
two types of studies (in^acts on growth-curve slopes versus in^acts on test-score levels), which leads to operational 
differences in the standard deviations used to. compute effect sizes and conceptual differences in the interpretation of 
effect sizes. 

^*The noncon^arability of the two types of study does not mean that one is inherently more correct than the 
other. They singly address different questions. 
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Chapter 6 

Conclusions and Implications 



This chapter reviews the constraints under which the present evaluation of the Acceler- 
ated Schools model was conducted, summarizes its main findings, and puts its conclusions and 
implications in a broader context. The chapter closes with recommendations for future research 
on Accelerated Schools and other education reforms. 

I. Constraints on the Study 

The goal of the present study was to provide an early, rigorous evaluation of the Acceler- 
ated Schools model’s effectiveness in improving student achievement in low-performing elemen- 
tary schools that serve students at risk of school failure. As discussed in earlier chapters, several 
factors played a key role in determining what questions the evaluation could address, how it 
could be designed, and how its findings should be interpreted. 

To complete the evaluation within several years, it was necessary to focus on early at- 
tempts to implement the Accelerated Schools model, which required that the research design be 
retrospective. The retrospective design, in turn, precluded use of random assignment, which is 
generally considered the best way to ensure that the observed effects of an intervention are due to 
the intervention rather than other factors. In addition, the study was designed to focus on the im- 
pacts of the model in elementary schools that had persevered long enough to implement its key 
components. Because the model takes several years to put into effect, the evaluation also required 
a lengthy follow-up period; only schools that had implemented the model early in its develop- 
ment — that is, in the early 1 990 s — could provide data for a sufficiently long follow-up period. 
Together the above constraints have important implications for what can be concluded fi-om the 
Study’s findings. 

II. Conclusions from the Study 

Implementation of the Accelerated Schools reform was a difficult, time-consuming proc- 
ess. But schools that stuck with the reform were able to improve the school environment appre- 
ciably, especially with respect to organizational culture and decision-making. These environ- 
mental improvements were followed by increases in students’ test scores, although it was not un- 
til the fifth year after the initiative was launched that average scores rose above their baseline 
levels by a statistically significant amount. 

The test score increases observed in this evaluation are of modest size by the conven- 
tional standards of evaluation research, but they are comparable to those found in the Tennessee 
class-size experiment, which examined an initiative that is widely considered to have been suc- 
cessful in improving the performance of elementary school students. 

The improvements in test scores found in the present study seem to have been concentrated 
among students who without the initiative would have scored in the middle of the baseline distribu- 
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tion for their school but who with the initiative scored in the upper baseline category. No test score 
gains were observed among students with weaker academic backgrounds. Though the reasons for 
the reform’s differential impacts on the two groups are unclear, one possible explanation is that stu- 
dent mobility was higher among lower-performing students, which would have decreased their ex- 
posme to the reform and thus presumably reduced the impacts that they experienced. 

Another important feature of the present findings is that the schools with the lowest aver- 
age baseline test scores relative to their state or national counterparts were the most likely to see 
large test score increases during the follow-up period. Thus, even though most of the impacts 
were concentrated among students who scored near the average for their school, these students 
typically scored below average statewide or nationally. Thus, judging fi-om the present sample of 
schools, the Accelerated Schools initiative appears to have been effective for students at risk of 
school failure. 

Lastly, it is important to note that the pattern of test score changes tracked the pattern of 
program implementation. During the first two years of implementation, when schools focused on 
changing their organizational cultures and decision-making processes, there was no systematic 
change in scores. In the third year, as the schools struggled to implement instructional changes 
with the limited guidance afforded by the early Accelerated Schools model, test scores dropped 
somewhat. In the final two years of the follow-up period, test scores gradually surpassed the 
baseline level. 

III. Limitations of the Study 

The present evaluation was as rigorous as possible given the constraints under which it 
was conducted. Nevertheless, it has important limitations, which in hun restrict the conclusions 
that cgn be drawn fi’om its findings. 

A. Drawing Causal Inferences 

There are several limits on the causal inferences — that is, conclusions about what caused 
the observed increases in student achievement — that can be drawn from the present study. As 
already discussed, it was not feasible to randomly assign each school (or each student) to a pro- 
gram or a control group. In the absence of random assignment, alternative explanations of the 
study’s findings cannot be definitively ruled out. 

For example, it is possible that local factors other than the Accelerated Schools reform 
(such as a change in principal, implementation of another educational initiative, or a sudden rise 
in teacher turnover) caused changes in test scores that distort the impact estimates. In an effort to 
rule out this threat, only schools where such changes were believed not to have occurred were 
selected for the study. Furthermore, while local factors might explain test score changes at a sin- 
gle school, they are less plausible explanations for the overall pattern of changes for the study’s 
sample of eight schools located in seven different states. Still, local events could have influenced 
the findings of the present study in unknown ways. 

In addition, it is possible that the abilities and backgrounds of students who attended the 
study schools during the follow-up period differ systematically fi’om those of students who at- 
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tended the schools-during the baseline period. To help guard against this possibility, only schools 
in which the composition of the student body was believed to have remained stable were in- 
cluded in the study. In addition, all analyses controlled statistically for student characteristics for 
which data were available. However, it is possible that there were shifts in student characteristics 
that were not accounted for in the analysis. 

Lastly, the fact that the estimated impacts of the Accelerated Schools model track the 
course of its implementation suggests that the initiative caused the impacts, though as explained 
above other factors may have played a causal role. This caveat notwithstanding, the similarity 
between the patterns of implementation and impacts, as well as their conformity to plausible hy- 
potheses about what might cause test scores to change, lend credibility to the inference that the 
reform caused the impacts that were observed. 

B. Generalizine Causal Inferences 

There are also limits on the ability to generahze the causal inferences that can be drawn from 
the present evaluation. One such limit stems from the fact that the study focused exclusively on ma- 
ture Accelerated Schools, which by definition were those that stuck with the reform long enough to 
implement it. Thus, the present findings suggest what can be achieved by a school that implements the 
Accelerated Schools model, but they do not indicate the likely outcome for all schools that might try 
to implement the model (some of which will do so and others of which will not). 

Another limit on the generalizability of the findings reflects the fact that it focused exclu- 
sively on early-generation Accelerated Schools. Because the reform model now places greater 
emphasis on early instructional changes, provides clearer guidelines for effecting these instruc- 
tional changes, and promotes the use of more extensive technical assistance by schools that are 
trying to implement the model, the impacts of current versions of the model might be larger and 
occur sooner than those observed in the present study. Also, the schools in the present study con- 
stitute neither a random nor a fully representative sample of all mature early-generation Acceler- 
ated Schools because they had to meet the research requirements to be selected. In fact, it is diffi- 
cult to characterize the population that these schools represent. Nevertheless, given the diverse 
geographic locations and student bodies included in the study, its findings probably reflect a wide 
range of situations. 

Lastly, because no statistically significant increases in average test scores were observed 
until five years after the reform was launched, the trend of the impacts on third-grade test scores 
in subsequent years cannot be predicted. Similarly, the study’s exclusive focus on the third grade 
does not allow for predictions about whether the impacts will grow, decay, or remain the same as 
students advance to later grades. 

IV. Future Research 

Given that the present findings (as those from any study) are neither definitive nor com- 
plete, this final section recommends next steps for future research. 
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A. Evaluate the Current Accelerated Schools Model 



Because the Accelerated Schools model has evolved in identifiable ways that might in- 
crease its ability to help schools and students, an important next step is to conduct an evaluation 
of the current model. Ideally, such an evaluation would include a large sample of elementary 
schools that represent a clearly definable population, focus on student performance in grades 
spanning the full elementary-school range, and use a random assignment design. 

B. Quantify Costs 

The Accelerated Schools model is generally regarded as a relatively inexpensive school 
reform. Most of the resources needed to implement it can be obtained by reallocating those that 
the school already has. The only additional resources are those needed to train faculty and staff 
and hire a part-time coach to facilitate implementation of the reform. 

To date, little precise information about the cost of implementing major school reforms, 
including Accelerated Schools, has been gathered. As discussed in Chapter 1 , perhaps the best 
cost analysis can be found is a study* of Accelerated Schools and two other well-known school 
reform models, Robert Slavin’s Success for All and James Comer’s School Development Pro- 
gram. According to the analysis. Accelerated Schools is less expensive than the other two mod- 
els. However, as the author acknowledges, this finding is based on limited data and untested as- 
sumptions, and much work on the costs of school reform remains to be done. 

C. Synthesize Research Findings 

Although attempts to develop and implement school reforms have been under way for 
well over a decade, little knowledge about the effectiveness of these initiatives has accumulated. 
The primary reason is the generally weak research designs used in past studies and the resulting 
lack of credibility of their findings.^ Recently, however, more rigorous designs have begun to be 
used in evaluations of education reforms. At the core of these new evaluations are two promising 
methodological approaches. 

The first ^proach — cluster random assignment, in which groups (such as schools) are 
randomly assigned — is a direct extension of classical randomized experiments, in which individu- 
als (such as students) are randomly assigned. In education research, cluster random assignment en- 
tails randomly assigning each school to a program group, which implements the reform under 
study, or a control group, which does not implement the reform or delays its implementation for a 
number of years.^ This approach has been used in two major studies of the School Development 
Program, one in Prince George’s County, Maryland, and the other in Chicago.'* The approach is 
also planned for use in a major evaluation of Success for All. 



'Barnett, 1996. 

^American Institutes for Research, 1999. 

^For a description and development of the theoretical properties of cluster random assignment, see Raudenbush 
(1997). For an enqjirical study of the statistical properties of the cluster random assignment of schools see Bloom, 
Bos, and Lee (1999). 

“*Cook et al., 2000; Cook, Hunt, and Miarphy, 2000. 
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A second methodological approach, value-added analysis of student-growth curves using 
hierarchical models, is a sophisticated design based on longitudinal student test data.^ This ap- 
proach has been used to evaluate the School Development Program in Detroit;^ Success for All, 
Accelerated Schools, and several other reform models in Memphis;’ and a major effort to insti- 
tute testing requirements for student promotion in Chicago.* 

As these and other rigorous studies of education reforms become available, it would be 
extremely valuable to develop a systematic process for synthesizing their results using methods 
from the field of meta-analysis.^ While it is difficult to compare and contrast findings from this 
growing body of studies (including the present one) because they use different measures and 
methods, the best way to ensure the future accumulation of knowledge from this body of re- 
search is to institute an ongoing systematic process for synthesizing findings across studies. 

D. Continue to Do Methodolo2ical Research 

The present evaluation of the Accelerated Schools model, based on interrupted time- 
series analysis, offers another avenue for studying education reforms. Specifically, combining 
interrupted time-series analysis with value-added analysis and hierarchical modeling can leverage 
the methodological benefits of each. MDRC is presently investigating the methodological proper- 
ties of this combined approach in three evaluations of school reform initiatives: Project GRAE>,” 
First Things First,*’ and Talent Development Schools.*’ Thus, the present study of Accelerated 
Schools represents the first in a series of efforts to develop better methods for measuring the im- 
pacts of education reforms where random assignment is not feasible. 



’Meyer (1997) provides a detailed discussion of value-added analysis. Bryk and Raudenbush (1992) offer a 
conqjrehensive discussion of hierarchical models. 

®Millsap et al., 2000. 

’Ross, Sanders, and Wright, 2000. 

®Jacob, Roderick, and Bryk, 2000. 

^Hedges and Olkin, 1985; Cooper and Hedges, 1994. 

'®01ejnik and Algina, 2000. 

"Ham, Doolittle, and Holton, 2000. 

'^MDRC, 2000a. 

"MDRC, 2000b. 
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Accelerated Schools Evaluation 

Appendix Table A.1 

Percentage of Reading Scores in the 
Lower Baseline Category for Each Follow-Up Year 



School 


Yearl 


Year 2 


Years 


Year 4 


Years 


A 


38 * 


28 


38 


32 


22 


B 


24 


24 


29 


35 ** 


32 • 


C 


25 


38 


20 


27 


N/A 


D 


29 


40 *** 


35 * 


32 


43 *** 


E 


20 


17 


32 


33 


17 


F 


27 


24 


17 * 


7 *** 


15 • 


G 


44 ••• 23 


27 


11 ** 


20 


H 


15 


21 


45 *** 


29 


29 


Average 


28 


27 


30 *** 


26 


25 






Accelerated Schools Evaluation 










Appendix Table A.2 








Percentage of Math Scores in the 
Lower Baseline Category for Each Follow-Up Year 




School 


Yearl 


Year 2 


Years 


Year 4 


Year 5 


A 


39 


23 


53 *• 


31 


22 


B 


28 


22 


28 


34 * 


26 


C 


17 


41 


33 


30 


N/A 


D 


31 


35 


22 


21 


28 


E 


13 


13 


14 


25 


30 


F 


25 


18 


25 


17 


22 


G 


38 * 


14 


12 * 


20 


6 *** 


H 


24 


28 


39 * 


35 


29 


Average 


27 


24 


28 


27 


23 



(continued) 
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Appendix Tables A.1 and A.2 (continued) 

SOURCE: MDRC calculations using student test scores and demographic data provided by each school. 



NOTES: The sample includes all third-grade students tested during the study period at each school. 

Estimates were adjusted using ordinary least squares to control for demographic characteristics of sample 
members. 

A two-tailed test was applied to differences between the baseline and follow-up test scores. Statistical 
significance levels are indicated as ***= 1 percent, **= 5 percent, .*= 10 percent. 

Findings in these tables represent equally weighted averages in which all schools received the same 
weight regardless of san^le sizes. 
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Accelerated Schools Evaluation 

Appendix Table A.3 

Percentage of Reading Scores in the 
Middle Baseline Category for Each Follow-Up Year 



School 


Year 1 


Year 2 


Years 


Year 4 


Years 


A 


44 


51 


51 


55 


52 


B 


46 


53 


56 


41 ** 


42 * 


c 


54 


46 


62 


56 


N/A 


D 


46 


48 


48 


41 


35 ** 


E 


40 


37 * 


54 


45 


50 


F 


31 *** 


39 ** 


30 *** 


25 *** 


22 *** 


G 


44 


52 


46 


59 


30 *** 


H 


56 


40 


38 ** 


49 


33 ** 


Average 


45 * 


46 * 


48 


46 


38 *** 
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Appendix Table A.4 








Percentage of Math Scores in the 






Middle Baseline Category for Each Follow-Up Year 




School 


Year 1 


Year 2 


Years 


Year 4 


Year 5 


A 


41 


33 ** 


42 


49 


35 


B 


47 


47 


46 


43 


50 


C 


47 


41 


51 


35 * 


N/A 


D 


57 


50 


58 


53 


49 


E 


42 


51 


48 


28 *** 


28 *** 


F 


53 


49 


47 


50 


33 *** 


G 


43 


44 


49 


43 


41 


H 


60 


47 


40 


49 


48 


Average 


49 


45 * 


47 


44 ** 


40 


(continued) 
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Appendix Tables A.3 and A.4 (continued) 

SOURCE: MDRC calculations using student test scores and demographic data provided by each school. 



NOTES: The sanqjle includes all third-grade students tested during the study period at each school. 

Estimates were adjusted using ordinary least squares to control for demographic characteristics of sample 
members. 

A two-tailed test was applied to differences between the baseline and follow-up test scores. Statistical 
significance levels are indicated as ***= 1 percent, **= 5 percent, *= 10 percent. 

Findings in these tables represent equally weighted averages in which all schools received the same 
weight regardless of sanq)le sizes. 
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Accelerated Schools Evaluation 
Appendix Table A.S 

Impacts on the Standard Deviation of Third-Grade Reading Scores 

for Each Follow-Up Year 









Effect Size 






School 


Year 1 


Year 2 


Years 


Year 4 


Years 


A 


1.21 * 


1.05 


0.98 


0.88 


1.17 


B 


1.02 


0.99 


0.74 *** 


1.07 


1.20 ** 


c 


1.13 


1.00 


0.93 


1.08 


N/A 


D 


0.96 


0.91 


1.01 


1.03 


1.23 ** 


E 


1.16 


1.16 


0.87 


1.11 


0.91 


F 


1.28 *** 


1.23 ** 


1.25 *** 


1.22 ** 


1.49 *** 


G 


1.06 


1.16 


1.17 


1.01 


1.13 


H 


0.87 


1.16 


1.19 * 


0.99 


1.31 *** 


Average 


1.07 * 


1.07 * 


1.03 


1.05 


1.13 *** 






Accelerated Schools Evaluation 










Appendix Table A.6 








Impacts on the Standard Deviation of Third-Grade Math Scores 
for Each Follow-Up Year 










Effect Size 






School 


Year 1 


Year 2 


Years 


Year 4 


Year 5 


A 


1.06 


1.23 ** 


1.00 


1.11 


1.42 ** 


B 


1.03 


0.91 


0.91 


0.96 


1.06 


C 


1.02 


1.05 


0.81 


1.13 


N/A 


D 


0.84 * 


0.98 


1.03 


1.08 


1.17 * 


E 


1.10 


1.15 


1.08 


1.43 *** 


1.26 ** 


F 


0.97 


0.92 


1.03 


0.95 


1.22 ** 


G 


1.05 


1.16 


0.99 


1.16 


0.99 


H 


0.89 


1.12 


1.31 *** 


1.11 


1.17 


Average 


1.02 


1.06 


1.03 


1.08 ** 


1.12 *** 



(continued) 
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Appendix Tables A.5 and A.6 (continued) 



SOURCE: MDRC calcxilations using student test scores and demographic data provided by each school. 

NOTES: The sample includes all third-grade students tested during the study period at each school. 

Estimates were adjusted using ordinary least squares to control for demographic characteristics of sample 
members. 

A two-tailed test was applied to differences between the baseline and follow-up test scores. Statistical 
significance levels are indicated as ***= 1 percent, **= 5 percent, *=10 percent. 

Findings in these tables represent equally weighted averages in which all schools received the same 
weight regardless of sample sizes. 
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Appendix B 

Qualitative Data Protocols 



Accelerated Schools Evaluation 



Appendix Table B.l 
Initial Contact Form 



School Name: Date Contacted: 

Who Contacted: 

Name and Title: 

Phone Number: - 

Level One Criteria Confirmed: 

Launch Date Confirmed: 

Grades Served Confirmed: 

School Information: 

Magnet School? No 

Yes Began (year): 

Principal (when current 
principal took position): 

% of school: 

School Demographics: 

Total Student Enrollment (1997-98): 

Total Grade Enrollment 1997-98): 

Number of 3’^*' Grade Classrooms (1997-98): 

Standardized Testing 

Do you do this? 

Which one? 

What year? 



Has the size of the grade class changed dramatically in the past ten years? If yes, how? 



Has the size of the student population dramatically changed in the past ten hears? If yes, how? 



(continued) 
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Appendix Table B.l (continued) 



Have demographics (i.e., ethnic, gender, or socio-economic characteristics) of the student 
population dramatically changed in the past ten years? How? 



Do student attend by neighborhood? By parental choice? District placement (e.g. busing)? 



Are standardized tests administered to students annually? 
If yes, which tests? 



What grades are tested?. 



Have the tests changed in the past ten years? 



If so, how?. 



May we contact the district? Yes. 



No (provide reason) 
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Appendix Table B.2 
Maturity Screen 

School 

Location: 



CADRES 

1 . What are your cadres? 

2. How often do cadres meet? 

3. Do they use the Inquiry Process to address their challenge areas? 

Yes:_ No: 

STEERING COMMITTEE 

4. Do you have a Steering Committee? 

Yes:_ No: 

5. Who is on the Steering Committee? 

6. How often does the Steering Committee meet? 

POWERFUL LEARNING 

7. Is powerful learning occurring in most of the classrooms and for all the students? “Powerful 
Learning” as the ASP describes in their literature? 

Yes:_ No: 

(continued) 
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Appendix Table B.2 (continued) 



AS CHANGES 

8. What changes (policy or curricular) have been implemented because of acceleration? 
OTHER ISSUES 

1 . How and why did you become an Accelerated School? 



2. Does the school work with an Accelerated Schools coach? How long has the coach worked 
with the school? Where are they based (school-based, district-based, Satellite Center)? 



3. Were there any major school initiatives in place at your school prior or during the adoption of 
acceleration? If yes, what were they? 



4. Review demographic and school size changes to see if change occurred gradually or 
dramatically. 
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Recent Publications on MDRC Projects 



Note: For works not published by MDRC, the publisher’s name is shown in parentheses. With a few exceptions, this 
list includes reports published by MDRC since 1999. A complete publications list is available from MDRC 
and on its Web site (www.mdrc.org), from which copies of MDRC’s publications can also be downloaded. 



Education Reform 

Accelerated Schools 

This study examines the implementation and impacts 
on achievement of the Accelerated Schools model, a 
whole-school reform targeted at at-risk students. 

Evaluating the Accelerated Schools Approach: A 
Look at Early Implementation and Impacts on 
Student Achievement in Eight Elementary Schools. 
2001. Howard Bloom, Sandra Ham, Laura Melton, 
Julienne O’Briea 

Project GRAD 

This evaluation examines Project GRAD, an 
education initiative targeted at urban schools and 
combining a number of proven or promising reforms. 

Building the Foundation for Improved Student 
Performance: The Pre-Curricular Phase of Project 
GRAD Newark. 2000. Sandra Ham, Fred Doolittle, 
Glee Ivory Holton. 

Career Academies 

The largest and most comprehensive evaluation of a 
school-to-woric initiative, this study examines a 
promising approach to high school restructuring and 
the school-to-work transition. 

Career Academies: Early Implementation Lessons 
from a ] 0-Site Evaluation. 1996. James Kemple, 
JoAnn Leah Rock. 

Career Academies: Communities of Support for 
Students and Teachers — Emerging Findings from 
a 10-Site Evaluation. 1997. James Kemple. 

Career Academies: Building Career Awareness and 
Work-Based Learning Activities Through Employer 
Partnerships. 1999. James Kenqjle, Susan 
Poglinco, Jason Snipes. 

SchooMo-Work Project 

A study of innovative programs that help students 
make the transition from school to work or cheers. 



Home-Grown Lessons: Innovative Programs Linking 
School and Work (Jossey-Bass Publishers). 1995. 
Edward Pauly, Hilary Kopp, Joshua Haimson. 
Home-Grown Progress: The Evolution of Innovative 
School-to-Work Programs. 1997. Rachel Pedraza, 
Edward Pauly, Hilary Kopp. 

Project Transition 

A demonstration program that tested a combination of 
school-based strategies to facilitate students’ 
transition from middle school to high school. 

Project Transition: Testing an Intervention to Help 
High School Freshmen Succeed. 1999. Janet Quint, 
Cynthia Miller, Jennifer Pastor, Rachel Cytron. 

Equity 2000 

Equity 2000 is a nationwide initiative sponsored by 
the College Board to improve low-income students’ 
access to college. The MDRC paper examines the 
inq)lementation of Equity 2000 in Milwaukee Public 
Schools. 

Getting to the Right Algebra: The Equity 2000 
Initiative in Milwaukee Public Schools. 1999. 
Sandra Ham, Erica Walker. 

Education for Adults and Families 

LILAA Initiative 

This study of the Literacy in Libraries Across 
America (LILAA) initiative explores the efforts of 
five adult literacy programs in public libraries to 
improve learner persistence. 

So I Made Up My Mind: Introducing a Study of Adult 
Learner Persistence in Library Literacy Programs. 
2000. John T. Comings, Sondra Cuban. 

'7 Did It for Myself: Studying Efforts to Increase 
Adult Learner Persistence in Library Literacy 
Programs. 2001. John Comings, Sondra Cuban, 
Johannes Bos, Catherine Taylor. 



Toyota Families in Schools 

A discussion of the factors that determine whether an 
impact analysis of a social program is feasible and 
warranted, using an evaluation of a new family 
literacy initiative as a case study. 

An Eyaluability Assessment of the Toyota Families in 
Schools Program, 2001. Janet Quint. 

Opening Doors to Earning Credentials 

An exploration of strategies for increasing low-wage 
workers’ access to and completion of community 
college programs. 

Opening Doors: Expanding Educational 
Opportunities for Low-Income Workers. 2001. 
Susan Golonka, Lisa Matus-Grossman. 

Effects of Welfare and Antipoverty 
Programs on Children 

Next Generation Project 

A collaboration among researchers at MDRC and 
several other leading research institutions focused on 
studying the effects of welfare, antipoverty, and 
enq)loyment policies on children and families. 

How Welfare and Work Policies Affect Children: A 
Synthesis of Research. 2001. Pamela Morris, Aletha 
Huston, Greg Duncan, Danielle Crosby, Johannes 
Bos. 

How Welfare and Work Policies Affect Employment 
and Income: A Synthesis of Research. 2001. Dan 
Bloom, Charles Michalopoulos. 

Minnesota Family Investment Program 

An evaluation of Minnesota’s pilot welfare reform 
initiative, which aims to encourage work, alleviate 
poverty, and reduce welfare dependence. 

Reforming Welfare and Rewarding Work: Final 
Report on the Minnesota Family Investment 
Program. Volume 2: Effects on Children. 2000. 

Lisa Gennetian, Cynthia Miller. 

Reforming Welfare and Rewarding Work: A 
Summary of the Final Report on the Minnesota 
Family Investment Program, 2000. Virginia Knox, 
Cynthia Miller, Lisa Gennetian. 

Canada’s Self-Sufficiency Project 

A test of the effectiveness of a temporary earnings 
supplement on the en^loyment and welfare receipt of 
public assistance recipients. 

The Self-Sufficiency Project at 36 Months: Effects on 
Children of a Program That Increased Parental 



Employment and Income (Social Research and 
Demonstration Corporation, Ottawa, Canada). 

2000. Pamela Morris, Charles Michalopoulos. 

National Evaluation of Welfare-to-Work 
Strategies 

Conceived and sponsored by the U.S. Department of 
Health and Human Services (HHS), with support 
from the U.S. Department of Education (ED), this is 
the largest-scale evaluation ever conducted of 
different strategies for moving people from welfare to 
employment. 

Do Mandatory Welfare-to-Work Programs Affect the 
Well-Being of Children? A Synthesis of Child Re- 
search Conducted as Part of the National Evalua- 
tion of Welfare-to-Work Strategies (HHS/ED). 
2000. Gayle Hamilton. 

Teen Parents on Welfare 

Teenage Parent Programs: A Synthesis of the Long- 
Term Effects of the New Chance Demonstration, 
Ohio *s Learning, Earning, and Parenting (LEAP) 
Program, and the Teenage Parent Demonstration 
(TPD). 1998. Robert Granger, Rachel Cytron. 

Ohio ’ s LEAP Program 

An evaluation of Ohio’s Learning, Earning, and 
Parenting (LEAP) Program, which uses financial 
incentives to encourage teenage parents on welfare to 
stay in or return to school. 

LEAP: Final Report on Ohio 's Welfare Initiative to 
Improve School Attendance Among Teenage 
Parents. 1997. Johannes Bos, Veronica Fellerath. 

New Chance Demonstration 

A test of a comprehensive program of services that 
seeks to in^rove the economic status and general 
well-being of a group of highly disadvantaged young 
women and their children. 

New Chance: Final Report on a Comprehensive 
Program for Young Mothers in Poverty and Their 
Children. 1997. Janet Quint, Johannes Bos, Denise 
Polit. 

Parenting Behavior in a Sample of Young Mothers in 
Poverty: Results of the New Chance Observational 
Study, 1998. Martha Zaslow, Carolyn Eldred, 
editors. 
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MDRC Working Papers on 
Research Methodology 

A new series of papers that explore alternative 
methods of examining the implementation and 
inq)acts of programs and policies. 

Building a Convincing Test of a Public Housing 
Employment Program Using Non-Experimental 
Methods: Planning for the Jobs-Plus 
Demonstration. 1999. Howalrd Bloom. 

Estimating Prograin Impacts on Student Achievement 
Using Short** Interrupted Time Series. 1999. 
Howard Bloom. 

Using Cluster Random Assignment to Measure 
Program Impacts: Statistical Implications for the 
Evaluation of Education Programs, 1999. Howard 
Bloom, Johannes Bos, Suk-Won Lee. 



Measuring the Impacts of Whole School Reforms: 
Methodological Lessons from an Evaluation of 
Accelerated Schools. 2001. Howard Bloom. 

The Politics of Random Assignment: Implementing 
Studies and Impacting Policy, 2000. Judith Gueron. 

Modeling the Performance ofWelfare-to-Work 
Programs: The Effects of Program Management 
and Services, Economic Environment, and Client 
Characteristics, 2001. Howard Bloom, Carolyn 
Hill, James Riccio. 

A Regression-Based Strategy for Defining Subgroups 
in a Social Experiment, 2001. James Kenq)le, Jason 
Snipes. 

Extending the Reach of Randomized Social 
Experiments: New Directions in Evaluations of 
American Welfare-to-Work and Employment 
Initiatives, 2001. James Riccio, Howard Bloom. 
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About MDRC 



The Manpower Demonstration Research Corporation (MDRC) is a nonprofit, 
nonpartisan social policy research organization. We are dedicated to learning what 
wprks to improve the well-being of low-income people. Through our research and 
the active communication of our findings, we seek to enhance the effectiveness of 
social policies and programs. MDRC was founded in 1974 and is located in New 
York City and San Francisco. 

MDRC’s current projects focus on welfare and economic security, education, and 
employment and community initiatives. Complementing our evaluations of a wide 
range of welfare reforms are new studies of supports for the working poor and 
emerging analyses of how programs affect children’s development and their 
families’ well-being. In the field of education, we are testing reforms aimed at 
improving the performance of public schools, especially in urban areas. Finally, our 
community projects are using innovative ^proaches to increase employment in 
low-income neighborhoods. 

Our projects are a mix of demonstrations — field tests of pro misin g program 
models — and evaluations of government and community initiatives, and we 
employ a wide range of methods to determine a program’s effects, including large- 
scale studies, surveys, case studies, and ethnogr^hies of individuals and families. 
We share the findings and lessons fi’om our work — including best practices for 
program operators — with a broad audience within the policy and practitioner 
community, as well as the general public and the media. 

Over the past quarter century, MDRC has worked in almost every state, all of the 
nation’s largest cities, and Canada. We conduct our projects in partnership with state 
and local governments, the federal government, public school systems, community 
organizations, and numerous private philanthropies. 
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